Conference PaperPDF Available

The Significance of Information Security Risk Assessments: Exploring the Consensus of Raters’ Perceptions of Probability and Severity

Authors:

Abstract

Identifying and assessing risks is vital in striving for adequate information security. The basis for the assessments is the probability and the severity of possible incidents affecting the confidentiality, integrity, and availability of information assets. However, assessing the probability and the severity of possible events is not straightforward. The objective of this paper is to explore the consensus of raters assessing the probability and the severity of information security incidents. Data collected through questionnaires are used to evaluate the consensus of 20 raters when assessing 105 information security incidents. The results indicate that the consensus of the raters is too low for the assessment results to provide a sound basis for decisions. In conclusion, better support is needed for assessing information security risks in order to reach the necessary consensus levels.
The Significance of Information Security Risk Assessments: Exploring the Consensus of Raters’
Perceptions of Probability and Severity
Jonas Hallberg, Johan Bengtsson, Niklas Hallberg, Henrik Karlzén, Teodor Sommestad
https://www.google.se/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahU
KEwil6JqXgfXXAhVkGZoKHQr_DbQQFggqMAA&url=http%3A%2F%2Fcsce.ucmss.com%2Fbooks%2FLF
S%2FCSREA2017%2FSAM9709.pdf&usg=AOvVaw02_b3yxM08i5hrG7zie6uR
Identifying and assessing risks is vital in striving for adequate information security. The basis for the
assessments is the probability and the severity of possible incidents affecting the confidentiality,
integrity, and availability of information assets. However, assessing the probability and the severity
of possible events is not straightforward. The objective of this paper is to explore the consensus of
raters assessing the probability and the severity of information security incidents. Data collected
through questionnaires are used to evaluate the consensus of 20 raters when assessing 105
information security incidents. The results indicate that the consensus of the raters is too low for the
assessment results to provide a sound basis for decisions. In conclusion, better support is needed for
assessing information security risks in order to reach the necessary consensus levels.
... However, developing the estimates and making decisions have typically required tedious and subjective manual scoring and ranking processes. Unfortunately, research suggests uncomfortably high scoring variance exists across teams that conduct such analyses [5]- [7]. ...
... Coopamootoo, et al. present a "design and reporting" toolkit that has nine key indicators of rigorous design for cybersecurity experiments [30]. Although the application of science in cybersecurity may not have advanced as far as we might have wished in the preceding decades (see [11], [23], [28], [29], [31] for some possible reasons why), some noteworthy experimental studies are in the literature [5][32]- [35]. ...
... A study by Hallberg et al. [29] explored inter-rater reliability and rater consensus when scoring the likelihood and severity of cyber security incidents. They found that "ratings of probability and severity are not reliable enough between raters to be considered a sound basis for the quantification of information security risks." ...
Conference Paper
Despite more than a decade of heightened focus on cybersecurity, cyber threats remain an ongoing and growing concern [1]-[3]. Stakeholders often perform cyber risk assessments in order to understand potential mission impacts due to cyber threats. One common approach to cyber risk assessment is event-based analysis which usually considers adverse events, effects, and paths through a system, then estimates the effort/likelihood and mission impact of such attacks. When conducted manually, this type of approach is labor-intensive, subjective, and does not scale well to complex systems. As an alternative, we present an automated capability-based risk assessment approach, compare it to manual event-based analysis approaches, describe its application to a notional space system ground segment, and discuss the results.
ResearchGate has not been able to resolve any references for this publication.