Conference PaperPDF Available

Aggregating Corporate Information Security Maturity Levels of Different Assets


Abstract and Figures

General Data Protection Regulation (GDPR) has not only a great influence on data protection but also on the area of information security especially with regard to Article 32. This article emphasizes the importance of having a process to regularly test, assess and evaluate the security. The measuring of information security however, involves overcoming many obstacles. The quality of information security can only be measured indirectly using metrics and Key Performance Indicators (KPIs), as no gold standard exist. Many studies are concerned with using metrics to get as close as possible to the status of information security but only a few focus on the comparison of information security metrics. This paper deals with aggregation types of corporate information security maturity levels from different assets in order to find out how the different aggregation functions effect the results and which conclusions can be drawn from them. The required model has already been developed by the authors and tested for applicability by means of case studies. In order to investigate the significance of the ranking from the comparison of the aggregation in more detail, this paper will try to work out in which way a maturity control should be aggregated in order to serve the company best in improving its security. This result will be helpful for all companies aiming to regularly assess and improve their security as requested by the GDPR. To verify the significance of the results with different sets, real information security data from a large international media and technology company has been used.
Content may be subject to copyright.
Aggregating Corporate Information Security
Maturity Levels of Different Assets
Michael Schmid1,2[000000023534313X]and
Sebastian Pape1[0000000208937856]
1Chair of Mobile Business & Multilateral Security,
Goethe University Frankfurt, Frankfurt, Germany
2Hubert Burda Media Holding KG, Muenchen, Germany
General Data Protection Regulation (GDPR) has not only a great influence
on data protection but also on the area of information security especially
with regard to Article 32. This article emphasizes the importance of having a
process to regularly test, assess and evaluate the security. The measuring of
information security however, involves overcoming many obstacles. The quality
of information security can only be measured indirectly using metrics and Key
Performance Indicators (KPIs), as no gold standard exist. Many studies are
concerned with using metrics to get as close as possible to the status of infor-
mation security but only a few focus on the comparison of information security
metrics. This paper deals with aggregation types of corporate information secu-
rity maturity levels from different assets in order to find out how the different
aggregation functions effect the results and which conclusions can be drawn
from them. The required model has already been developed by the authors
and tested for applicability by means of case studies. In order to investigate
the significance of the ranking from the comparison of the aggregation in more
detail, this paper will try to work out in which way a maturity control should
be aggregated in order to serve the company best in improving its security.
This result will be helpful for all companies aiming to regularly assess and
improve their security as requested by the GDPR. To verify the significance
of the results with different sets, real information security data from a large
international media and technology company has been used.
Information security
Information security management
Aggregation functions
Information security controls
maturity model ·Security maturity model ·Security metrics framework
1 Introduction
Approximately 18 months ago the General Data Protection Regulation (GDPR)
containing requirements regarding the processing of personal data of individuals
became operative.
Accepter at IFIP SC 2019
Copyright Springer, Cham
2 Michael Schmid and Sebastian Pape
The GDPR states that organizations must adopt appropriate policies, pro-
cedures and processes to protect the personal data they hold. Article 32 of the
GDPR specifically requires organizations to ensure confidentiality, integrity, avail-
ability and resilience (core principles of the information security) of processing
systems and services, and to implement a process for regularly testing, assessing
and evaluating the effectiveness (e.g. with KPIs) of technical and organizational
measures for ensuring secure processing [
]. Thus, in addition to presenting a
state of the art security level, this article emphasizes the importance of a process
for regularly testing, assessing and evaluating the security. However, it does not
provide detailed guidance on how to achieve these goals.
It is difficult to judge whether the security level is sufficient from a management
perspective. Managers often act according to the maxim ’minimal effort maxi-
mum success’, since the budget is usually limited. Of course, this also applies
to the area of information security and varies depending on the industry and
the self-perception of IT security within it. This is justifiable from an economic
point of view, but it has an influence on how information security is dealt with in
the company. In this situation, it is important to create transparency regarding
the state of information security, within an organization to determine how good
the process is, as well as in comparison to other companies operating in the
same environment. This transparency can be used to demonstrate/ensure that
(information) security does not suffer from budget constraints.
An established way to monitor and steer the information security is the imple-
mentation of an information security management system (ISMS). With the most
popular standard in this field, ISO/IEC 27001 [
], it is possible to manage the
information security in a company through the ISO-controls. An effective ISMS
that conforms to ISO/IEC 27001 meets all requirements of GDPR’s article 32.
The information security status of an environment like a company is a very
individual observation [
]. To estimate the actual status of information security
normally metrics or key performance indicators (KPI) are taken into account [
The information gathering of these KPIs is usually done through different techni-
cal or organizational metrics of a company. Using KPI/Metric/Maturity for the
status of information security is only an indicator of improvement or deterioration
since there is unfortunately no gold standard for this [
]. It would be very complex
and expensive to first collect or generate these KPIs for this evaluation. It is
important therefore, to work with the data/metrics already available and no need
for further data collection. In this context, it should not go unmentioned that an-
other standard exists in this environment, the ISO/IEC 27701 [
]. This standard
deals with how to establish and run a Privacy Information Management System
(PIMS) that adds Personally Identifiable Information (PII) security protection to
an existing ISMS. In order to assess the status of information security as well as
the quality of the process, mostly a maturity model is used. A common method
for the assessment of the maturity is the COBIT control maturity model from
the ISACA framework [
]. With the help of this model it is possible to assess
the goodness of the ISO-controls on a 0 to 5 scale. The assessment supports
the improvement of the organization’s security and delivers the management
Aggregating Corporate Information Security Maturity Levels 3
Fig. 1:
Exemplary holding structure with different assets and control maturity for
perspective in the fulfillment of regulatory requirements.
With the maturity level, the manager has a relatively good overall view of the
status of information security. However, this is usually a very aggregated view of
the status, as a company will operate different types of IT systems/applications
to support its business process. The information assets worth protecting (e.g.
customer data, trade secrets, source code, etc.) are not only processed or stored
on one IT system, but on several. As a consequence, the maturity level may differ
between systems. Therefore, many companies not only collect a maturity level
for the whole company, but also a maturity level per system for each control [
An ISO control such as A.12.6.1 (Vulnerability Management) will only be able
to reflect a combined value from several IT systems/applications. That’s why,
different values exist for different assets per ISO control (see figure. 1).
In order to derive a KPI from the assets’ control maturity level or use them
as input for existing approaches [
] [
], the questions arises how they can be
meaningfully aggregated.
RQ1: How can maturity levels for one control be meaningfully aggregated
across different assets?
Different aggregation types can not only influence the outcome of the approach,
but also influence the managers which security controls should be improved.
RQ2: How would a manager’s optimization strategy depend on the different
aggregation methods?
And finally, it’s equally important to consider the aggregation’s influence on
the final result of the algorithm.
RQ3: How much does the outcome of a holistic approach actually change
depending on different aggregation types?
4 Michael Schmid and Sebastian Pape
To examine this research question, we first discuss different types of aggrega-
tion for maturity levels. In the next step, for each of the aggregations we derive
possible security managers’ optimization strategies in order to establish which
control to improve next. For a reality check, we examine asset’s maturity levels
from real company data to check if our assumptions are realistic. As a final step,
we also use real companies’ maturity levels to examine how much the outcome
of [24] would be changed by applying a different aggregation.
The remainder of this work is structured as follows: In Sect. 2 we give a brief
overview of related work. Section 3 describes our methodology how we developed
our approach for each research question shown in Sect. 4. Our results are shown
in 5 and discussed in Sect. 6, respectively Sect. 7.
2 Background and Related Work
In addition to the differences in the assessment of information security, all
assessment procedures also have in common that the ratings of the maturity
level and the weighting of weights are not allocated to a common overall value in
the sense of an ’information security score’. It is, therefore up to the evaluator
to carry out the respective evaluation, as he or she is forced to choose between
these two quantitative aspects of the evaluation, e. g. the ratings on the one hand
and the weighting on the other [
]. Savola [
] discussed a broader approach to
finding a metrics which can be used in the field of different security disciplines
like management and engineering practices. In contrast to this, the works of
Boehme [
] and Anderson [
] deal more with the economic impact of investments in
information security. There are also other models that deal with the measurement
of information security using maturity levels e.g. the Information security maturity
model (ISMM) [
] and the Open Information Security Maturity Model (O-
ISM3) [
]. ISMM is intended as a tool to evaluate the ability of organizations to
meet the objectives of security and O-ISM3 aims to ensure that security processes
operate at a level consistent with business requirements. However, both models
refer more to the process level than to the asset level. The focus of this work is
to compare the different aggregation types of maturity within an industry. This
could later lead to a monetary assessment of information security or maturity.
2.1 Aggregation types
Unfortunately, the precise process of how to aggregate maturity levels is neither
well documented nor comprehensively studied or understood (from a psychological
perspective), so most of this labor is done by rule-of-thumb [
]. As mentioned,
our approach varies between four aggregation types - namely the minimum,
maximum, average and median - to compare their different potential impacts
on decision making. Regarding the two measures of central tendency (average,
median), strengths and weaknesses have been discussed in scientific literature.
Averages are strongly influenced by extreme values. In our context, this could lead
to an over- or underestimate of control maturity. In contrast, the median is not
Aggregating Corporate Information Security Maturity Levels 5
skewed by extreme values, consequently running the risk of overestimating control
maturity [
]. The opposite can be the case when there are multiple non-values
(e.g. zeros) in a data sample, as laid out by Anderson et al. [
]. The relative
position of average and median differs in skewed distributions. A distribution
skewed to the left will lead to a smaller median compared to the average, while
a right-skewed distribution reverses the relation [
]. Overall, it makes sense to
include both measures of central tendencies in our analysis to compensate for
weaknesses and bias. The minimum and maximum further alleviate potential
misrepresentations of control maturity, as they provide the numerical range
of scores and expose potential outliers [
]. Logically, both measures are most
sensitive to outliers in a data set but are nevertheless useful in our analysis when
used in combination with the measures of central tendency.
2.2 Aggregation of security metrics
Although the domain of security metrics has been covered by a number of
authors [
], only limited work on the area of metrics aggregation has been carried
out. Ramos et al. [
] provided a detailed survey on models for quantifying
networks resilience to attacks. The authors used stochastic techniques and attack
graphs to map the possible routes an attacker could take to compromise a system.
Abraham et al. [
] discussed the challenges faced by practitioners in the field
of security measurements and highlighted the need to develop a mechanism for
quantifying the overall security of all the systems on the network. The authors
proposed a predictive framework that uses stochastic techniques based on attack
graphs and incorporated temporal factors relating to the vulnerabilities such
as availability of patch and exploits predicting the future state of the system.
Cheng et al. [
] proposed a model for aggregating security metrics using Common
Vulnerability Scoring System (CVSS) base metrics to estimate the exploitability of
the vulnerabilities. Homer et al. [
] and Beck at al. [
] proposed a mathematical
security model for aggregating vulnerabilities in risks in enterprise networks
based on attack graphs. An aggregated numeric value was assigned to show the
likelihood of a vulnerability being exploited by an attacker.
3 Research Methodology
The general aim of our approach is to determine which effect the different
aggregation types of the maturity control of assets have on the information
security of the companies. In order to do this it is important to create transparency
around the state of information security. The method should take into account
the different requirements of the different research questions set out in chapter 1.
We derive the different aggregation methods in the next subsection for our
approach, then determine the proper algorithm and finally describe the data
collection of our approach.
6 Michael Schmid and Sebastian Pape
3.1 Different Aggregation Functions
First, we examine which functions are suitable to verify the approach described
above. As shown in Tab. 2.1 and 2.2 with the different aggregation functions e.g.
average, median, minimum and maximum it is possible to form a single summary
value from a group of data. The challenge now is to find the right aggregation
functions to support the approach provided. These aggregation functions have in
common that they can represent the impact of decisions by information security
managers, each type in its own way. The hypotheses provide an outlook how
information security managers might behave in terms of aggregation.
3.2 Data Collection
It would be very complex and expensive to first collect or generate these KPIs
for this evaluation. It is important to use data/metrics already available (e.g.
information security maturity level). To test the above approach it is necessary to
set up the model and verify it with real data. We need a maturity assessment of
the ISO/IEC controls and to weight and aggregate them according to the specific
industry. We focused on the eCommerce industry for the following reasons:
Available data from a large range of companies
Excellent data quality and validity
High actuality of the existing data
Very good know-how available in the expert assessment of the industry
We collected data from Hubert Burda Media (HBM), an international media and
technology company (over 12,500 employees, more than 2.5 billion annual sales,
represented in over 20 countries). This group is divided into several business
units that serve various business areas (including print magazines, online portals,
eCommerce etc.). The business units consist of over 250 individual companies
with about 30 of them being in the eCommerce industry. Each subsidiary operates
independently of the parent corporation. There is a profit center structure, so the
group acts as a company for entrepreneurs and the managing directors have the
freedom to invest money in information security and to choose the appropriate
level of security. We will briefly describe how this data is collected before going
into more detail on the data used for the comparison. Each individual company
in the group operates its own Information Security Management System (ISMS)
in accordance with ISO/IEC 27001, which is managed by an Information Security
Officer (ISO) on site and managed by a central unit in the holding company.
As part of the evaluation of the ISMS, the maturity level of the respective ISO
27001 controls is ascertained - very granularly at the asset level (application,
web-server, CRM etc.). The maturity level is collected/updated regularly once a
year as part of a follow-up procedure.
3.3 Algorithm Method Selection
Taking all requirements of the method into account, a previously developed
approach from Schmid&Pape [
] is applicable. The primary objective of this
Aggregating Corporate Information Security Maturity Levels 7
approach was to show how to use the analytic hierarchy process (AHP) to compare
the information security controls of a level of maturity within an industry in
order to rank different companies. The AHP is one of the most commonly
used Multiple Criteria Decision Methods (MCDM), combining subjective and
personal preferences in the information security assessment process [
]. It allows
a structured comparison of the information security maturity level of companies
with respect to an industry [
] and to obtain a ranking [
]. This allows the
definition of a separate weighting of information security metrics for each industry
with respect to their specifics while using a standardized approach based on the
maturity levels of the ISO/IEC 27001 controls.
To achieve the aim of this paper it is necessary to calculate the control maturity
of the assets with different aggregation types such as: minimum, maximum,
average or median. This shows how strong the characteristics of the individual
aggregation types are in comparison to the real data. Out of this, the first
indicators can then be derived to clarify which effect the aggregation types
have on the information security for individual companies. The following chapter
describes the implementation of the approach for each of the 3 research questions.
4 Discussion of Different Aggregations
As outlined in the previous chapter the different aggregation functions have a
very likely a different outcome when it comes down comparing them with each
other. Among other things, this chapter will describe the different characteristics
of the aggregation functions as well as the effects of the various IT assets of a
company and how they affect the results. A vivid example with real world data
illustrates how the various aggregations affect the final result and ultimately the
behaviour of those responsible for information security.
4.1 General Aggregation Functions
The great advantage of the aggregation functions average, median, minimum
and maximum is that by aggregating (key) figures differences can be identified
in the results and thus comparisons can be made. These could be a strength or
weakness per each aggregation type. In contrast to this, there is no difference in
the comparison of the results for the aggregation functions sum, range and count,
for example. A further advantage of the four aggregation functions mentioned
above is the adaptability of these types to a different number of values. They
work nicely even if each company has a different number of assets considered.
This makes it possible to derive different scenarios for the comparison.
4.2 Derived Optimization Strategies
If the results of the different aggregation functions are compared with each other,
different optimization strategies can be derived in the end. This is particularly
important for those who are responsible for information security. Due to the
8 Michael Schmid and Sebastian Pape
Asset Company1 Company2 Company3 Company4 Company5
1 4 03 3 4
2 4 224
342 3
4 1 1
5 0
Table 1:
Maturity levels of different collective assets for the ISO-control A.12.6.1 from
five companies
Aggregation Company1 Company2 Company3 Company4 Company5
average 2.6 0 2.3 2.25 4
median 4 0 2 2.5 4
minimum 0 0 2 1 4
maximum 4 0 3 3 4
Table 2: Maturity level results from different aggregation functions
different aggregations, it is possible that different optimization possibilities can
be shown in the evaluation of information security. The information security
manager can then decide which optimization strategy/aggregation function brings
him the most benefit. If we take a closer look at the 4 aggregation functions
mentioned above and examine them for the possible outcome, we obtain the
following hypotheses:
improve only the worst value (weakest chain, can make sense),
maximum improve only the best value (is this desirable?),
average improve any value (probably the easiest ones first) and
may lead to a really two-fold security level with
services and n+1
2secure services.
As next step we validate these hypotheses using an example with real world data.
4.3 Example with real world data
In order to compare the results of the different aggregation functions we need
real data. Chapter 3.2 describes how these real data, in this case the COBIT
maturity, are collected. For a concrete example we use the maturity level for a
specific ISO-Control (here A.12.6.1 ’Management of Technical Vulnerabilities’)
because this control focuses on an IT asset. As an example, we use data from
five companies and their various IT assets (see table 1).
Based on this data, the calculations of the four different aggregation functions
are now performed (see table 2) for the five companies. The colored cells highlight
the aggregation functions and the maturity levels used. These exemplary calcula-
tions are based on the maturity levels of companies with different IT assets. A
company uses many different IT assets to support its core and support processes.
The next chapter examines these different types of IT assets in more detail.
Aggregating Corporate Information Security Maturity Levels 9
4.4 More Complex Aggregations
In order to steer manager’s optimization strategy one needs to integrate weightings
for the different assets. This leads to the problem that many approaches, e.g.
] only work with a fixed number of assets. Considering only a fixed set of
assets for each domain would narrow the defined scope, thus it should be possible
to still evaluate a different number of assets. Conclusion: Define most important
assets and their weighting and build an asset class for all remaining assets. This
way, at least the impact of the manager’s optimization strategies is more limited
and only usable among the assets within the ’special class’. Arising Question:
How to derive the priorities for all the classes?
When considering the core business processes for an eCommerce company, the
web presence, a merchandise management system and a customer management
system are normally expected. For this stage, we examined the prevailing situation
of the IT assets used by 25 eCommerce companies from HBM and evaluated
them. Almost all eCommerce companies had a web sever (24), a database server
(24), an ERP system (22) and a CRM system (20). Further IT assets, which
did not have such a high frequency were mail servers (14), file servers (14), dev
servers (12), git (9), ftp servers (7), etc. This also coincides with the assumption
resulting from the core business processes. Resulting from this the core IT assets
of an eCommerce company, a web sever, a database server, an ERP system and
a CRM system were selected.
Only considering these core IT assets would not reflect the overall picture
of an eCommerce company. In order to have a comprehensive picture we also
need the assets that are used in the IT department (e.g. file server, dev server,
ftp server etc.). We have combined these IT assets into one collective asset for
the comprehensive picture). In a further step, this collective asset, or better
the maturity level, is calculated or evaluated using various aggregation types
(minimum, maximum, average, median). In combination with the 4 core assets,
aggregated values of the collective assets are included in the calculation as 5th
assets (with 20%). This can provide the first insights as to whether a certain
aggregation method might influence the units or sub-companies decision, hence
which control should be improved next.
4.5 Priorization of Asset Classes
The core IT assets are equally important (e.g. 25% for each) at the moment. An
interesting question would be e.g. how much more important is the web server of
an eCommerce company compared to the ERP system? It would be necessary
to add an additional layer of prioritization in order to differentiate between the
differing control requirements. In order to implement this we could use the CIA
triad model which encompasses a triangle of tension between the three principles
Confidentiality, Integrity and Availability. When applied to our use case, the
principles of importance vary between control objectives and is represented by
a score for the CIA principles according to their importance for these control
10 Michael Schmid and Sebastian Pape
Table 3: Combined GAP of Core assets and AHP Score
AHP Verbal
Score description
9 Extreme
8 preference
7 Very strong
6 preference
5 Strong
4 preference
3 Moderate
2 preference
1 Equal preference
(a) Fundamental AHP
9 22.22 - 25.00 Extreme
preference8 19.45 - 22.21
7 16.67 - 19.44 Very strong
preference6 13.89 - 16.66
5 11.12 - 13.88 Strong
preference4 08.34 - 11.11
3 05.56 - 08.33 Moderate
preference2 02.78 - 05.55
1 00.00 - 02.77 Equal
(b) AHP Score vs. GAP of the CIA dif-
objectives. This would provide for an extension of the approach by the CIA values
of the individual assets. In order to do this, we need the CIA evaluation per IT
asset. The information (e.g. customer data, contracts etc.) is stored or processed
on an IT asset. It allows conclusions to be drawn as to how this asset should be
treated in terms of confidentiality, integrity and availability. This means that
there is at least one information asset per asset, but usually several information
assets per asset, which are evaluated according to the CIA criteria with a 3-
step classification (normal, advanced and high). A web server will, for example,
process or even store information assets such as customer data, bank details, etc.
If the information values ’customer data’ and ’bank details’ for a web server are
uniformly evaluated for confidentiality, integrity and availability according to a
given system, this can be set in relation to an ERP system with the information
values ’purchasing conditions’ and ’master data’. A further step was needed to
convert our CIA data to pairwise comparisons on our AHP score, as depicted in
table 3a. We define a factor of equal importance regarding the CIA triad of all
four core assets as a proportion percentage of 25% each. Consequently, we can
conduct pairwise comparisons related to the proportion gaps in our data, which
are then normalized based on the AHP preference score i.e. equal importance
(AHP score: 1) is expressed by tiny differences in proportion to percentage of
smaller than 2.77%, while the highest order of relative importance (AHP score:
9) means a difference of 25% in proportion to percentage (see table 3b).
5 Results of the holistic approach considering different
aggregation types
The aim of this paper is to find out which effects the different aggregation
functions have on the results and which conclusions can be drawn from them.
Aggregating Corporate Information Security Maturity Levels 11
Aggregation/Proportion Company1 Company2 Company3 Company4 Company5
Average 15.4% 7.7% 30.8% 30.8% 15.4%
Median 12.6% 12.6% 27.4% 34.9% 32.0%
Minimum 10.0% 10.0% 40.0% 20.0% 20.0%
Maximum 22.2% 11.1% 22.2% 22.2% 22.2%
Table 4:
Comparison of different aggregation types from 5 companies only for control
The different aggregation functions can not only influence the outcome of the
approach, but also influence the manager’s decision as to the order in which
control’s maturity levels should be increased. They can influence the manager’s
optimization strategy depending on the different aggregation functions. At present,
the maturity levels have not yet been examined with a view to optimization.
5.1 Results of Aggregated Maturity Levels
The AHP was used to compare the maturity levels in order to work out how a
maturity control should be determined to best serve the company in improving
its security with reference to the first research question [
]. Table 4 shows a
comparison of results with different aggregation types from five companies only
for control A.12.6.1 ’Management of Technical Vulnerabilities’. Because this
control is asset-based, this value is composed of different IT assets that were
calculated with each of the 4 different aggregation types.
As expected, Company 2 is very weakly developed if the raw data in table 1
is considered. Company 1 is also quite clearly recognizable with regard to the
minimum and maximum. Company 3 has the highest proportion concerning the
minimum (40.0%). The results show that a detailed look at Company 5 would
be worthwhile, as the largest fluctuations between average and median (15.4% -
32.0%) can be observed here.
If we now abstract this comparison to a higher level, e.g. no longer to the
control level but to control category level, the results should no longer fluctuate
greatly. In the case of control categories, we are concentrating only on the most
important ones for the eCommerce industry. The weighting of the respective
control categories can be seen from the results of the AHP [
]. ’A.14’ (System
Acquisition, Development and Maintenance) is the most important for the eCom-
merce industry with 16.5%, followed by ’A.17’ (Information Security Aspects
of Business Continuity Management) with 14.7% and then ’A.12’ (Operations
security) with 9.5%. Table 5 shows how the individual eCommerce companies
weighting is compared with each other and the four different aggregation types
for ’A.12’ Operations security are compared in detail.
The rows total up to 9.5% because it is the ratio of ’A.12’ weighting in contrast
to the overall control categories. The distribution of values within an aggregation
type per company is specified in brackets. The differences are marginal but a
closer inspection more pronounced differences can be observed at the control
12 Michael Schmid and Sebastian Pape
Aggregation Company1 Company2 Company3 Company4 Company5
Average 1.7% (17.9%) 1.2% (12.6%) 2.3% (24.2%) 2.1% (22.1%) 2.2% (23.1%)
Median 1.6% (16.8%) 1.7% (17.9%) 2.4% (25.3%) 1.9% (20.0%) 1.9% (20.0%)
Minimum 1.4% (14.7%) 1.2% (12.6%) 2.8% (29.5%) 2.1% (22.1%) 2.0% (21.0%)
Maximum 1.8% (18.9%) 1.3% (13.7%) 1.7% (17.9%) 1.6% (16.8%) 3.1% (32.6%)
Table 5:
Comparison (Proportion) of different aggregation types from 5 companies for
control category A.12
Aggregation/Proportion Company1 Company2 Company3 Company4 Company5
Average 16.7% (4.) 15.4% (5.) 19.8% (1.) 18.3% (3.) 19.5% (2.)
Median 16.7% (4.) 16.3% (5.) 19.8% (1.) 18.8% (2.) 18.1% (3.)
Minimum 16.6% (4.) 14.6% (5.) 21.3% (1.) 18.7% (2.) 18.5% (3.)
Maximum 17.5% (2.) 15.6% (5.) 16.1% (4.) 16.2% (3.) 24.2% (1.)
Table 6:
Comparison of different aggregation types from 5 companies for the complete
ISO/IEC 27001
level and therefore tendencies are recognizable. Company 3 has again the highest
proportion concerning the minimum (29.5%)
The last comparison in this environment is the application of the four different
aggregation types to the complete controls of Annex A of ISO/IEC 27001. This is
ultimately the highest expected level of aggregation of this approach. It is to be
expected that the results will no longer differ so much from each other. Table 6
shows the results of the comparison.
The rows total up only to 89.9% because 11.1% is a ’measure of the error due
to inconsistency’ which is provided by the AHP. The ranking within all companies
is specified in brackets. Concerning the outcome of the comparison, Company 5
stands out with a high value for maximum aggregation (24.2%) and Company
1 looks very stable concerning the different aggregation types. Generally, the
minimum does not fluctuate as much as the maximum. Company 1 to 3 have
no high fluctuation in common and concerning Company 3 there is not a lot of
variance can be observed.
5.2 Results of Priorization the Asset
The descriptive statistic of HBMs information asset presence is used to begin
with the set of four core assets, namely web server (24), database server (24),
ERP system (22) and CRM system (20). Besides, computing our input scores as
well as defining our priorities for sub criteria level requires the processing of the
CIA inputs. The summarizing statistic is presented in table 7 below:
All CIA scores are summed up for each asset and divided by the total number
(see table 8). The lowest sum resulted from the CRM asset with 100, and is hence
our base value.
Concerning the priorization of asset classes table 9 shows a pairwise com-
parison of the core assets from one eCommerce company. The deviation is then
Aggregating Corporate Information Security Maturity Levels 13
Company Information Asset for Confidentiality Integrity Availability Sum of CIA
Company 1 Web-Server 2 2 3 7
Web server 3 3 3 9
Web server 3 3 2 8
Web server 2 3 2 7
Web server 3 3 2 8
Database server 2 2 2 6
Database server 2 2 2 6
ERP system 2 2 2 6
ERP system 2 2 2 6
ERP system 2 2 2 6
ERP system 2 2 2 6
CRM system 2 2 2 6
CRM system 2 2 2 6
CRM system 1 2 2 5
CRM system 1 2 2 5
Company 2 ... ... ... ... ...
Table 7: CIA of information assets from different IT assets of one company
Asset CIA sum distribution
WEB 156 32.5%
ERP 104 25.0%
DB 120 21.7%
CRM 100 20.8%
Table 8: Distribution of Assets
transformed into the AHP scores with the help of the intervals from the GAP of
core assets (see table 3b). It is clear that the biggest difference lies between the
web server and the CRM system (11.7%) and the smallest difference between
the CRM system and the database server (0.7%). With the help of this score it
is possible to weight the core assets based on their CIA assessment and process
them with the AHP.
6 Discussion
Based on these results, we discuss the main findings as follows. The results show
that it is possible to elaborate differences in the assessment and comparison of IT
assets with the help of different aggregation types. The main goal of this paper, to
assist managers in how they can improve their information security by comparing
different aggregated information security maturity levels on asset level has shown
several outcomes. The results show that a certain type of aggregation affects a
company when trying to improve its maturity levels (see table 4). Company 1
and 2 would improve first the collective assets with a low control maturity if
a minimum aggregation is used. If the aggregation function maximum is used
14 Michael Schmid and Sebastian Pape
Sub criteria A Sub criteria B A/B Deviation Score
WEB ERP A +7.25% 3
WEB DB A +10.8% 4
WEB CRM A +11.7% 5
ERP DB A +2.3% 1
ERP CRM A +4.1% 2
DB CRM A +0.7% 1
Table 9: AHP Comparison with core assets
Company 3 would try to improve one collective asset in order to maximize only
one control maturity (see table 5). Concerning the big picture in table 6 the
ranking of the companies differs only for Company 1 and 3. Company 1 has
already very high control maturities, so it is not as easy for them to improve.
Company 3 almost a very homogenous control maturity thats why the would
probably improve only one collective asstets if the maximum aggregation is
chosen. The other companies are more or less stable concerning the ranking, e.g.
Company 2 does not changes at all.
With the help of the CIA pritorisation is possible to first weight and then
aggregated the different IT systems and applications with each other (see table 9).
The results show hat for an eCommerce company it is obvious that the web server
is more important than the ERP-System in supporting the business processes.
6.1 Limitations
Maturity levels are not assessed automatically but by each of the individual com-
panies’ information security officer (ISO). Therefore, there may be discrepancies
in the way the maturity levels are understood and assessed. This is clearly a
limitation of any approach based on security maturity levels, but it might limit
the informative value of the collected maturity levels. Moreover, the maturity
levels are reported to the management and they result in a key performance
indicator (KPI) for security for that specific unit. Thus, it can be assumed that
each ISO has an interest in having a good evaluation. Therefore, ISOs might
be tempted to assess the maturity levels more optimistically or to limit the
scope of the information security management system in order to achieve better
evaluations more easily. A common understanding of the different maturity levels
is already established by guidelines and manuals provided to the ISOs (of HBM).
This could be expanded further in order to reach a better understanding for the
assessment of control maturity levels. Furthermore, deviations can be addressed
if the companies are (externally) audited from time to time to double check the
maturity levels.
7 Conclusion and Future Work
The discussion of how an overall score for a maturity level for security controls
across different assets shows that the aggregation is an important tool needed to
Aggregating Corporate Information Security Maturity Levels 15
distinguish how the information security managers would optimize information
security. In practice it makes a big difference which aggregation is used because
it could lead to optimizing only the control maturity levels which are easily
reachable. The defined priorization is necessary in order not to depend too much
on the different kind of optimization strategies of the managers. This way, it can
be steered more directly where the security should be enhanced and it probably
also reflects better the current security level of companies. This approach is
a helpful result for all companies aiming to regularly assess and improve their
security as requested by the GDPR in order to ensure the confidentiality, integrity,
availability and resilience of IT assets and evaluating the effectiveness of the
technical and organizational measures for ensuring the security process.
As future work the outcome with other approaches could be compared to sen
how the aggregation has changes the influence. Additionally, one might need to
find other ways to prioritize the different controls, since in this case it was easy
since it’s one of the AHPs natural properties. Further investigations have to been
carried out in order to clarify the validity of the control maturity levels because
of the containing bias. Additional work could also be carried out to check validity
of scope in order to measure any changes in the results after the metrics have
been introduced.
Abbas Ahmed, R.K.: Security Metrics and the Risks: An Overview. International
Journal of Computer Trends and Technology 41(2), 106–112 (2016)
Abraham, S., Nair, S.: A Predictive Framework For Cyber Security Analytics Using
Attack Graphs. International journal of Computer Networks & Communications
Ahmed, Y., Naqvi, S., Josephs, M.: Aggregation of security metrics for decision
making: A reference architecture. ACM International Conference Proceeding Series
Anderson, R., Barton, C., B¨ohme, R., Clayton, R., van Eeten, M.J., Levi, M.,
Moore, T., Savage, S.: Measuring the cost of cybercrime. In: The Economics of
Information Security and Privacy, pp. 265–300. Springer Berlin Heidelberg (2013)
Anderson, R., Barton, C., Rainer, B., Clayton, R., Ga, C., Grasso, T., Levi,
M., Moore, T., Vasek, M.: Measuring the Changing Cost of Cybercrime Our
Framework for Analysing the Costs of Cybercrime. In: Workshop on the Economics
of Information Security (WEIS). pp. 1–32 (2019)
Beck, A., Rass, S.: Using neural networks to aid CVSS risk aggregation - An
empirically validated approach. Journal of Innovation in Digital Ecosystems (2016)
Bland, M.: Estimating Mean and Standard Deviation from the Sample Size, Three
Quartiles, Minimum, and Maximum. International Journal of Statistics in Medical
Research (2015)
ohme, R.: Security metrics and security investment models. In: Lecture Notes in
Computer Science (including subseries Lecture Notes in Artificial Intelligence and
Lecture Notes in Bioinformatics). vol. 6434 LNCS, pp. 10–24 (2010)
Cheng, P., Wang, L., Jajodia, S., Singhal, A.: Aggregating CVSS base scores for
semantics-rich network security metrics. In: Proceedings of the IEEE Symposium
on Reliable Distributed Systems (2012)
16 Michael Schmid and Sebastian Pape
Doane, D.P., Seward, L.E.: Applied Statistics in Business and Economics. McGraw-
Hill Higher Education (2016)
Gordon, L.a., Loeb, M.P.: The economics of information security investment. ACM
Transactions on Information and System Security 5(4), 438–457 (2002)
Homer, J., Zhang, S., Ou, X., Schmidt, D., Du, Y., Rajagopalan, S.R., Singhal,
A.: Aggregating vulnerability metrics in enterprise networks using attack graphs.
Journal of Computer Security (2013)
ISACA: COBIT 5: A business framework for Governance and Management of
Enterprise IT (2012)
ISO/IEC 27001: Information Technology - Security Techniques - Information Se-
curity Management Systems - Requirements. International Organization for Stan-
dardization (2013)
ISO/IEC 27701: Security techniques - Extension to ISO/IEC 27001 and ISO/IEC
27002 for privacy information management - Requirements and guidelines. Interna-
tional Organization for Standardization (2019)
Khajouei, H., Kazemi, M., Moosavirad, S.H.: Ranking information security controls
by using fuzzy analytic hierarchy process. Information Systems and e-Business
Management 15(1) (2017)
Lee, M.c.: Information Security Risk Analysis Methods and Research Trends : AHP
and Fuzzy Comprehensive Method. International Journal of Computer Science &
Information Technology (IJCSIT) 6(February), 29–45 (2014)
Manikandan, S.: Measures of central tendency: Median and mode. Journal of
Pharmacology and Pharmacotherapeutics (2011)
Nasser, A.A.: Measuring the Information Security Maturity of Enterprises under
Uncertainty Using Fuzzy AHP. I.J. Information Technology and Computer Science
4(April), 10–25 (2018)
Ramos, A., Lazar, M., Filho, R.H., Rodrigues, J.J.: Model-Based Quantitative
Network Security Metrics: A Survey (2017)
Rudolph, M., Schwarz, R.: Security Indicators – A State of the Art Survey Public
Report. FhG IESE VII(043) (2012)
Saleh, M.: Information security maturity model. International Journal of Computer
Science and Security (IJCSS) 5, 21 (01 2011)
Savola, R.M.: Towards a taxonomy for information security metrics. In: Proceedings
of the ACM Conference on Computer and Communications Security. pp. 28–30
Schmid, M., Pape, S.: A structured comparison of the corporate information
security. In:
Systems Security and Privacy Protection - 34th
} {
11 International Conference,
2019, Lisbon, Portugal, June 25-27, 2019,
Proceedings (2019)
Schmitz, C., Pape, S.: LiSRA: Lightweight Security Risk Assessment for Decision
Support in Information Security. Computers & Security (2019)
Syamsuddin, I., Hwang, J.: The application of AHP to evaluate information security
policy decision making. International Journal of Simulation: Systems, Science and
Technology 10(4), 46–50 (2009)
Vinet, L., Zhedanov, A.: A ’missing’ family of classical orthogonal polynomials.
Journal of Physics A: Mathematical and Theoretical 44(8) (2011)
ResearchGate has not been able to resolve any citations for this publication.
Conference Paper
Full-text available
In 2012 we presented the first systematic study of the costs of cybercrime. In this paper, we report what has changed in the seven years since. The period has seen major platform evolution, with the mobile phone replacing the PC and laptop as the consumer terminal of choice, with Android replacing Windows, and with many services moving to the cloud. The use of social networks has become extremely widespread. The executive summary is that about half of all property crime, by volume and by value, is now online. We hypothe- sised in 2012 that this might be so; it is now established by multiple victimisation studies. Many cybercrime patterns appear to be fairly stable, but there are some interesting changes. Payment fraud, for example, has more than doubled in value but has fallen slightly as a proportion of payment value; the payment system has simply become bigger, and slightly more efficient. Several new cybercrimes are significant enough to mention, including business email compromise and crimes involving cryptocurrencies. The move to the cloud means that system misconfiguration may now be responsible for as many breaches as phishing. Some companies have suffered large losses as a side-effect of denial-of-service worms released by state actors, such as NotPetya; we have to take a view on whether they count as cybercrime. The infrastructure supporting cybercrime, such as botnets, continues to evolve, and specific crimes such as premium-rate phone scams have evolved some interesting variants. The over- all picture is the same as in 2012: traditional offences that are now technically ‘computer crimes’ such as tax and welfare fraud cost the typical citizen in the low hundreds of Eu- ros/dollars a year; payment frauds and similar offences, where the modus operandi has been completely changed by computers, cost in the tens; while the new computer crimes cost in the tens of cents. Defending against the platforms used to support the latter two types of crime cost citizens in the tens of dollars. Our conclusions remain broadly the same as in 2012: it would be economically rational to spend less in anticipation of cybercrime (on antivirus, firewalls, etc.) and more on response. We are particularly bad at prosecuting criminals who operate infrastructure that other wrongdoers exploit. Given the growing realisation among policymakers that crime hasn’t been falling over the past decade, merely moving online, we might reasonably hope for better funded and coordinated law-enforcement action.
Full-text available
Information security risk assessment frameworks support decision-makers in assessing and understanding the risks their organisation is exposed to. However, there is a lack of lightweight approaches. Most existing frameworks require security-related information that are not available and that are very challenging to gather. So they are not suitable in practice, especially for small and medium-sized enterprises (SMEs) who often lack in data and in security knowledge. On the other hand, other explicit SME approaches have far less informative value than the proposed framework. Moreover, many approaches only provide extensive process descriptions that are challenging for SMEs. In order to overcome this challenge, we propose LiSRA, a lightweight, domain-specific framework to support information security decision-making. It is designed with a two-sided input where domain experts initially provide domain-specific information (e.g. attack scenarios for a specific domain), whereupon users can focus on specifying their security practices and organisational characteristics by entering information that many organisations have already collected. This information is then linked to attack paths and to the corresponding adverse impacts in order to finally assess the total risk. Moreover, LiSRA can be used to get transparent recommendations for future security activities and presents detailed insights on the mitigating effects of each recommendation. The security activities are being evaluated taking into account the security activities already in place, and also considering the dependencies between multiple overlapping activities that can be of complementary, substitutive or dependent nature. Both aspects are ignored by most existing evaluation approaches which can lead to an over-investment in security. A prototype has been implemented, and the applicability of the framework has been evaluated with performance and robustness analyses and with initial qualitative evaluations.
Conference Paper
Full-text available
Generally, measuring the information security maturity is the first step to build a knowledge information security management system in an organization. Unfortunately, it is not possible to measure information security directly. Thus, in order to get an estimate, one has to find reliable measurements. One way to assess information security is by applying a maturity model and assess the level of controls. This does not need to be equivalent to the level of security. Nevertheless, evaluating the level of information security maturity in companies has been a major challenge for years. Although many studies have been conducted to address these challenges, there is still a lack of research to properly analyze these assessments. The primary objective of this study is to show how to use the analytic hierarchy process (AHP) to compare the information security controls’ level of maturity within an industry in order to rank different companies. To validate the approach of this study, we used real information security data from a large international media and technology company.
Full-text available
Generally, measuring the Information Security maturity(ISM) is the first step to build a new knowledge information security management system in an organization. Knowing the ISM level helps organizations decide the type of protection strategies and policies will be taken and their priorities to strengthen their competitive ability. One of the possible ways to solve the problem is a using multiple criteria decision-making (MCDM) methodology. Analytic hierarchy process (AHP) is one of the most commonly used MCDM methods, which combines subjective and personal preferences in the information security assessment process. However, the AHP involves human subjectivity, which introduces vagueness type of uncertainty and requires the use of decision-making under those uncertainties. In this paper, the IS maturity is based on hierarchical multilevel information security gap analysis model for ISO 27001:2013 security standard. The concept of fuzzy set is applied to Analytic Hierarchical Process (AHP) to propose a model for measuring organizations IS maturity under uncertain environment. Using fuzzy AHP approach helps determine more efficiently importance weights of factors and indicators, especially deal with imprecise and uncertain expert comparison judgments. A case study is used to illustrate the better new method for IS evaluation
Full-text available
measuring information security is difficult; it is difficult to have one metrics that covers all types of devices. Security metrics is a standard used for measuring any organization's security. Good metrics are needed for analysts to answer many security related questions. Effective measurement and reporting are required to improve effectiveness and efficiency of controls, and ensure strategic alignment in an objective, reliable, and efficient manner. This paper provides an overview of the security metrics and its definition, standards, advantages, types, problems, taxonomies, risk assessment methods and also classifies the security metrics and explains its risks.
Full-text available
Managing risks in large information infrastructures is often tied to inevitable simplification of the system, to make a risk analysis feasible. One common way of “compacting” matters for efficient decision making is to aggregate vulnerabilities and risks identified for distinct components into an overall risk measure related to an entire subsystem and the system as a whole. Traditionally, this aggregation is done pessimistically by taking the overall risk as the maximum of all individual risks, following the heuristic understanding that the “security chain” is only as strong as its weakest link. As that method is quite wasteful of information, this work proposes a new approach, which uses neural networks to resemble human expert’s decision making in the same regard. To validate the concept, we conducted an empirical study on human expert’s risk assessments, and trained several candidate networks on the empirical data to identify the best approximation to the opinions in our expert group.
Full-text available
Information security can be achieved by implementing a set of appropriate controls. However, identifying and selecting the most effective information security controls in organizations have been major challenges for years. Although many studies have been done to address these challenges, there is still lack of research to rank these controls. In this study, a fuzzy Analytic Hierarchy Process was used to prioritize and select effective managerial domains and control objectives in information security controls. In this research, the process of implementing ISO 27001 Information Security in National Iranian Oil Products Distribution Company was selected. According to results, the access control, information systems acquisition, development and maintenance have the highest priorities among the information security controls in managerial domains. On the other hand, the business continuity management and asset management have the lowest priorities among the studied information security controls. Furthermore, it was found that among 39 control objectives, the user access management and third party service delivery management have the highest and lowest priorities, respectively.
Technical Report
Full-text available
Measurement is one of the foundations of sound engineering practices, because—as Tom DeMarco put it—you cannot control what you can’t measure. This principle should also apply to software security engineering. However, providing useful metrics or at least indicators for characterizing the security properties of a software system is surprisingly challenging. The research community is well aware of the urgent need for security metrics, and it has put significant research effort into this field. Numerous qualitative and quantitative security measures have been proposed in the scientific literature, but few of them found wide-spread adoption by practitioners. Due to the significant body of work, it has become increasingly difficult to overlook the state of the art in specifying, determining, comparing, or predicting security qualities. This report surveys the published work on security indicators. In the context of this survey, a security indicator is understood as an observable characteristic that correlates with a desired security property. Our survey covers current research into qualitative and quantitative security indicators as well as applied key performance indicators and security standards. We developed a uniform classification scheme for categorizing and comparing the indicators that we elicited. Based on this classification, our survey reveals trends and deficiencies in security research and security practice. It also suggests explanations for the apparent difficulties in providing meaningful security indicators. Moreover, our classification can guide practitioners to adequate methods for the specification of security requirements and for the measurement of relevant security attributes of their products and processes.
Conference Paper
Existing security technologies play a significant role in protecting enterprise systems but they are no longer enough on their own given the number of successful cyberattacks against businesses and the sophistication of the tactics used by attackers to bypass the security defences. Security measurement is different to security monitoring in the sense that it provides a means to quantify the security of the systems while security monitoring helps in identifying abnormal events and does not measure the actual state of an infrastructure's security. The goal of enterprise security metrics is to enable understanding of the overall security using measurements to guide decision making. In this paper we present a reference architecture for aggregating the measurement values from the different components of the system in order to enable stakeholders to see the overall security state of their enterprise systems and to assist with decision making. This will provide a newer dimension to security management by shifting from security monitoring to security measurement.
Network security metrics (NSMs) based on models allow to quantitatively evaluate the overall resilience of networked systems against attacks. For that reason, such metrics are of great importance to the security-related decision-making process of organizations. Considering that over the past two decades several model-based quantitative NSMs have been proposed, this article presents a deep survey of the state-of-the-art of these proposals. First, to distinguish the security metrics described in this survey from other types of security metrics, an overview of security metrics, in general, and their classifications is presented. Then, a detailed review of the main existing model-based quantitative NSMs is provided, along with their advantages and disadvantages. Finally, this survey is concluded with an in-depth discussion on relevant characteristics of the surveyed proposals and open research issues of the topic.