Conference Paper

How Context Impacts Vulnerability Severity: An Analysis of Product-Specific CVSS Scores

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

Article
Full-text available
Cybercrime affects companies worldwide, costing millions of dollars annually. The constant increase of threats and vulnerabilities raises the need to handle vulnerabilities in a prioritized manner. This prioritization can be achieved through Common Vulnerability Scoring System (CVSS), typically used to assign a score to a vulnerability. However, there is a temporal mismatch between the vulnerability finding and score assignment, which motivates the development of approaches to aid in this aspect. We explore the use of Natural Language Processing (NLP) models in CVSS score prediction given vulnerability descriptions. We start by creating a vulnerability dataset from the National Vulnerability Database (NVD). Then, we combine text pre-processing and vocabulary addition to improve the model accuracy and interpret its prediction reasoning by assessing word importance, via Shapley values. Experiments show that the combination of Lemmatization and 5,000-word addition is optimal for DistilBERT, the outperforming model in our experiments of the NLP methods, achieving state-of-the-art results. Furthermore, specific events (such as an attack on a known software) tend to influence model prediction, which may hinder CVSS prediction. Combining Lemmatization with vocabulary addition mitigates this effect, contributing to increased accuracy. Finally, binary classes benefit the most from pre-processing techniques, particularly when one class is much more prominent than the other. Our work demonstrates that DistilBERT is a state-of-the-art model for CVSS prediction, demonstrating the applicability of deep learning approaches to aid in vulnerability handling. The code and data are available at https://github.com/Joana-Cabral/CVSS_Prediction .
Conference Paper
Full-text available
When a new computer security vulnerability is publicly disclosed, only a textual description of it is available. Cybersecurity experts later provide an analysis of the severity of the vulnerability using the Common Vulnerability Scoring System (CVSS). Specifically, the different characteristics of the vulnerability are summarized into a vector (consisting of a set of metrics), from which a severity score is computed. However, because of the high number of vulnerabilities disclosed everyday this process requires lot of manpower, and several days may pass before a vulnerability is analyzed. We propose to leverage recent advances in the field of Natural Language Processing (NLP) to determine the CVSS vector and the associated severity score of a vulnerability from its textual description in an explainable manner. To this purpose, we trained multiple BERT classifiers, one for each metric composing the CVSS vector. Experimental results show that our trained classifiers are able to determine the value of the metrics of the CVSS vector with high accuracy. The severity score computed from the predicted CVSS vector is also very close to the real severity score attributed by a human expert. For explainability purpose, gradient-based input saliency method was used to determine the most relevant input words for a given prediction made by our classifiers. Often, the top relevant words include terms in agreement with the rationales of a human cybersecurity expert, making the explanation comprehensible for end-users.
Article
Full-text available
Assessing the risks of software vulnerabilities is a key process of software development and security management. This assessment requires to consider multiple factors (technical features, operational environment, involved assets, status of the vulnerability lifecycle, etc.) and may depend from the assessor’s knowledge and skills. In this work, we tackle with an important part of this problem by measuring the accuracy of technical vulnerability assessments by assessors with different level and type of knowledge. We report an experiment to compare how accurately students with different technical education and security professionals are able to assess the severity of software vulnerabilities with the Common Vulnerability Scoring System (v3) industry methodology. Our results could be useful for increasing awareness about the intrinsic subtleties of vulnerability risk assessment and possibly better compliance with regulations. With respect to academic education, professional training and human resources selections our work suggests that measuring the effects of knowledge and expertise on the accuracy of software security assessments is feasible albeit not easy.
Article
Full-text available
In this paper we introduce an objective method for CVSS score calculation. CVSS is a well known and mostly used method for giving priority to software vulnerabilities. Currently it is being calculated by some slightly subjective methods which require enough skill and knowledge. This research shows how we can benefit from natural language description of vulnerabilities for CVSS calculation. The data that were used for implementation and evaluation of the proposed models consists of the available CVE vulnerability descriptions and their corresponding CVSS scores from the OSVDB database. First, feature vectors were extracted using text mining tools and techniques, and then the SVM and Random-Forest algorithms as well as fuzzy systems were examined to predict the concerned CVSS scores. In spite of the fact that SVM and Random-Forest are mostly used and trusted methods in prediction, results of this research bear a witness that using fuzzy systems can give comparable and even better results. In addition, implementation of the fuzzy based system is much easier and faster. Although so far, there have been so little efforts in using the information embedded in textual materials regarding vulnerabilities, this research shows that it will be valuable to utilize them in systems security establishment.
Article
Full-text available
Requirements engineering, a vital component in successful project development, often neglects sufficient attention to security concerns. Further, industry lacks a useful model for incorporating security requirements into project development. Studies show that upfront attention to security saves the economy billions of dollars. Industry is thus in need of a model to examine security and quality requirements in the development stages of the production lifecycle.In this paper, we examine a methodology for both eliciting and prioritizing security requirements on a development project within an organization. We present a model developed by the Software Engineering Institute's Networked Systems Survivability (NSS) Program, and then examine two case studies where the model was applied to a client system. The NSS Program continues to develop this useful model, which has proven effective in helping an organization understand its security posture.
Article
Scoring vulnerabilities is a hard task, and we build standards for this purpose: the Common Vulnerability Scoring System (CVSS) used by NIST is the oldest of them. It was invented by Peter Mell and Karen Scarfone, among others, to assess severity.1
Conference Paper
The use of learning-based techniques to achieve automated software vulnerability detection has been of longstanding interest within the software security domain. These data-driven solutions are enabled by large software vulnerability datasets used for training and benchmarking. However, we observe that the quality of the data powering these solutions is currently ill-considered, hindering the reliability and value of produced outcomes. Whilst awareness of software vulnerability data preparation challenges is growing, there has been little investigation into the potential negative impacts of software vulnerability data quality. For instance, we lack confirmation that vulnerability labels are correct or consistent. Our study seeks to address such shortcomings by inspecting five inherent data quality attributes for four state-of-the-art software vulnerability datasets and the subsequent impacts that issues can have on software vulnerability prediction models. Surprisingly, we found that all the analyzed datasets exhibit some data quality problems. In particular, we found 20-71% of vulnerability labels to be inaccurate in real-world datasets, and 17-99% of data points were duplicated. We observed that these issues could cause significant impacts on downstream models, either preventing effective model training or inflating benchmark performance. We advocate for the need to overcome such challenges. Our findings will enable better consideration and assessment of software vulnerability data quality in the future.
Article
Vulnerability databases are vital sources of information on emergent software security concerns. Security professionals, from system administrators to developers to researchers, heavily depend on these databases to track vulnerabilities and analyze security trends. How reliable and accurate are these databases though In this paper, we explore this question with the National Vulnerability Database (NVD), the U.S. governments repository of vulnerability information that arguably serves as the industry standard. Through a systematic investigation, we uncover inconsistent or incomplete data in the NVD that can impact its practical uses, affecting information such as the vulnerability publication dates, names of vendors and products affected, vulnerability severity scores, and vulnerability type categorizations. We explore the extent of these discrepancies and identify methods for automated corrections. Finally, we demonstrate the impact that these data issues can pose by comparing analyses using the original and our rectified versions of the NVD. Ultimately, our investigation of the NVD not only produces an improved source of vulnerability information, but also provides important insights and guidance for the security community on the curation and use of such data sources.
Conference Paper
Cyber-physical systems are a crucial part of many infrastructure or production systems, and are spreading into other domains as part of the IoT (Internet-of-Things) wave. As cyber-physical systems act on the physical world, attacks could have severe consequences. At the same time, cyber-physical systems can be attacked like other IT systems. So it is essential that developers consider security during the design phase of software, to design adequate security protection for the system. This fact requires a structured security analysis right from the beginning. The initial input of such a security analysis is a system overview, e.g. in form of an architecture. It is a challenging task to provide the appropriate abstraction level of the system that allows identifying security threats and weaknesses. In the present paper, we describe a pattern that assists software developers in creating an architecture which captures the relevant elements for a security analysis. The interfaces of components may not only be accessible for authorized entities, but also for attackers. Therefore, we specify different interface types which enables one to identify relevant attacks for a specific interface type. We first present the solution part of our pattern as a meta-model, for which we then provide guidelines for its instantiation. As an example, we instantiate the pattern for a typical automation and control system. Last, we evaluate the suitability of our pattern by discussing how typical threats could be mapped to the different interface types.
Article
(U. S.) Rule-based policies for mitigating software risk suggest using the CVSS score to measure the risk of an individual vulnerability and act accordingly. A key issue is whether the 'danger' score does actually match the risk of exploitation in the wild, and if and how such a score could be improved. To address this question, we propose using a case-control study methodology similar to the procedure used to link lung cancer and smoking in the 1950s. A case-control study allows the researcher to draw conclusions on the relation between some risk factor (e. g., smoking) and an effect (e. g., cancer) by looking backward at the cases (e. g., patients) and comparing them with controls (e. g., randomly selected patients with similar characteristics). The methodology allows us to quantify the risk reduction achievable by acting on the risk factor. We illustrate the methodology by using publicly available data on vulnerabilities, exploits, and exploits in the wild to (1) evaluate the performances of the current risk factor in the industry, the CVSS base score; (2) determine whether it can be improved by considering additional factors such the existence of a proof-of-concept exploit, or of an exploit in the black markets. Our analysis reveals that (a) fixing a vulnerability just because it was assigned a high CVSS score is equivalent to randomly picking vulnerabilities to fix; (b) the existence of proof-of-concept exploits is a significantly better risk factor; (c) fixing in response to exploit presence in black markets yields the largest risk reduction.
Towards the detection of inconsistencies in public security vulnerability reports
  • Ying Dong
  • Wenbo Guo
  • Yueqi Chen
  • Dong Ying