A preview of this full-text is provided by Springer Nature.
Content available from Empirical Software Engineering
This content is subject to copyright. Terms and conditions apply.
A preview of the PDF is not available
Comparing effectiveness and efficiency of Interactive Application Security Testing (IAST) and Runtime Application Self-Protection (RASP) tools in a large java-based system
Abstract and Figures
Context
Security resources are scarce, and practitioners can benefit from guidance in the effective and efficient usage of tools and techniques to detect and prevent the exploitation of software vulnerabilities. Interactive Application Security Testing (IAST) is a vulnerability detection tool that combines static and dynamic testing using sensor modules and agents. Runtime Application Self-Protection (RASP) tools monitor an application’s behavior and block attempts to exploit existing vulnerabilities in a running application. IAST and RASP have not often been compared to well-established counterparts, such as Dynamic Application Security Testing (DAST), Static Application Security Testing (SAST), and penetration testing, particularly in the context of a large system.
Objective
The goal of this research is to aid practitioners in making informed choices about the use of Interactive Application Security Testing (IAST) and Runtime Application Self-Protection (RASP) tools through an analysis of their effectiveness and efficiency in comparison with other vulnerability detection and exploit prevention techniques and tools.
Methods
We apply IAST and RASP on OpenMRS, an open-source Java-based online medical records web application. We compare the efficiency and effectiveness of IAST and RASP with techniques applied on OpenMRS in prior work: Systematic (SMPT) and Exploratory (EMPT) Manual Penetration Testing techniques and SAST and DAST tools. We measure efficiency and effectiveness in terms of the number and type of vulnerabilities detected and prevented per hour.
Results
In the context of a large, enterprise-scale web application, our study shows that IAST performed second best in both efficiency and effectiveness. IAST’s efficiency (2.14 Vulnerabilities per Hour (Vph)) is second to EMPT’s (2.22 VpH). IAST found 91 (8%) unique vulnerabilities not found by other tools and techniques, with SAST finding 823 (71%) unique vulnerabilities. Regarding effectiveness, IAST detected 8 of the Top-10 OWASP web application security risks, compared to 9 by SMPT and 7 by EMPT, DAST, and SAST. Our results indicate that RASP only prevents Injection attacks in OpenMRS.
Conclusion
IAST is an efficient and effective tool that complements other vulnerability detection tools and techniques. RASP does not replace vulnerability detection but can prevent the exploitation of existing injection vulnerabilities in a running application.
Figures - available from: Empirical Software Engineering
This content is subject to copyright. Terms and conditions apply.
ResearchGate has not been able to resolve any citations for this publication.
As software applications continue to become more complex and attractive to cyber-attackers, enhancing resilience against cyber threats becomes essential. Aiming to provide more robust solutions, different approaches were proposed for vulnerability detection in different stages of the application life-cycle. This article explores three main approaches to application security: Static Application Security Testing (SAST), Dynamic Application Security Testing (DAST), and Software Composition Analysis (SCA). The analysis conducted in this work is focused on open-source solutions while considering commercial solutions to show contrast in the approaches taken and to better illustrate the different options available. It proposes a baseline comparison model to help evaluate and select the best solutions, using comparison criteria that are based on community standards. This work also identifies future opportunities for application security, highlighting some of the key challenges that still need to be addressed in order to fully protect against emerging threats, and proposes a workflow that combines the identified tools to be used for vulnerability assessments.
The use of static analysis security testing (SAST) tools has been increasing in recent years. However, previous studies have shown that, when shipped to end users such as development or security teams, the findings of these tools are often unsatisfying. Users report high numbers of false positives or long analysis times, making the tools unusable in the daily workflow. To address this, SAST tool creators provide a wide range of configuration options, such as customization of rules through domain-specific languages or specification of the application-specific analysis scope. In this paper, we study the configuration space of selected existing SAST tools when used within the integrated development environment (IDE). We focus on the configuration options that impact three dimensions, for which a trade-off is unavoidable, i.e., precision, recall, and analysis runtime. We perform a between-subjects user study with 40 users from multiple development and security teams - to our knowledge, the largest population for this kind of user study in the software engineering community. The results show that users who configure SAST tools are more effective in resolving security vulnerabilities detected by the tools than those using the default configuration. Based on post-study interviews, we identify common strategies that users have while configuring the SAST tools to provide further insights for tool creators. Finally, an evaluation of the configuration options of two commercial SAST tools, Fortify and CheckMarx, reveals that a quarter of the users do not understand the configuration options provided. The configuration options that are found most useful relate to the analysis scope.
Context
Applying vulnerability detection techniques is one of many tasks using the limited resources of a software project.
Objective
The goal of this research is to assist managers and other decision-makers in making informed choices about the use of software vulnerability detection techniques through an empirical study of the efficiency and effectiveness of four techniques on a Java-based web application.
Method
We apply four different categories of vulnerability detection techniques – systematic manual penetration testing (SMPT), exploratory manual penetration testing (EMPT), dynamic application security testing (DAST), and static application security testing (SAST) – to an open-source medical records system.
Results
We found the most vulnerabilities using SAST. However, EMPT found more severe vulnerabilities. With each technique, we found unique vulnerabilities not found using the other techniques. The efficiency of manual techniques (EMPT, SMPT) was comparable to or better than the efficiency of automated techniques (DAST, SAST) in terms of Vulnerabilities per Hour (VpH).
Conclusions
The vulnerability detection technique practitioners should select may vary based on the goals and available resources of the project. If the goal of an organization is to find “all” vulnerabilities in a project, they need to use as many techniques as their resources allow.
Context: Static Application Security Testing (SAST) and Runtime Application Security Protection (RASP) are important and complementary techniques used for detecting and enforcing application-level security policies in web applications.
Inquiry: The current state of the art, however, does not allow a safe and efficient combination of SAST and RASP based on a shared set of security policies, forcing developers to reimplement and maintain the same policies and their enforcement code in both tools.
Approach: In this work, we present a novel technique for deriving SAST from an existing RASP mechanism by using a two-phase abstract interpretation approach in the SAST component that avoids duplicating the effort of specifying security policies and implementing their semantics. The RASP mechanism enforces security policies by instrumenting a base program to trap security-relevant operations and execute the required policy enforcement code. The static analysis of security policies is then obtained from the RASP mechanism by first statically analyzing the base program without any traps. The results of this first phase are used in a second phase to detect trapped operations and abstractly execute the associated and unaltered RASP policy enforcement code.
Knowledge: Splitting the analysis into two phases enables running each phase with a specific analysis configuration, rendering the static analysis approach tractable while maintaining sufficient precision.
Grounding: We validate the applicability of our two-phase analysis approach by using it to both dynamically enforce and statically detect a range of security policies found in related work. Our experiments suggest that our two-phase analysis can enable faster and more precise policy violation detection compared to analyzing the full instrumented application under a single analysis configuration.
Importance: Deriving a SAST component from a RASP mechanism enables equivalent semantics for the security policies across the static and dynamic contexts in which policies are verified during the software development lifecycle. Moreover, our two-phase abstract interpretation approach does not require RASP developers to reimplement the enforcement code for static analysis.
