Conference Paper

Efficient Monitoring of Library Call Invocation

To read the full-text of this research, you can request a copy directly from the authors.


The ability to monitor when user code invokes a library function offers numerous advantages. For example, during black-box testing of code, high-level control-flow integrity (CFI) checking, run-time access-control policy enforcement and so on. However, for this technique to be useful it must be efficient and able to function even when the target application is provided only as a statically linked executable. In an earlier paper we demonstrated how library calls may be intercepted using wrappers. But this approach works only with dynamically linked code and requires labour-intensive processing of the function arguments. Under the current scheme, each library - either dynamically or statically linked - constitutes a separate code-region. At any point in time, only pages belonging to that one region are marked as executable, so when code branches to a page outside the `home' region, it will land in a non-executable page, a fault will occur and the kernel will take over. By adding suitable code to the kernel, we can determine (a) whether the call should go ahead, (b) whether the arguments are acceptable and (c) ensure that the kernel will be informed when the code returns from the called function. In this paper we present our technique by analyzing the interception of a known exploit of the NGINX server. We show that our mechanism can detect and contain the attack and discuss the performance overheads of our mechanism.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

The ability to analyze software systems without access to the source code, offers many advantages including the detection of vulnerabilities so that they may be fixed before an adversary can exploit them in a zero day attack. This type of analysis also has an important role in education as it allows students to use their imagination and creativity in the exploration process. In this paper, we use two techniques for black-box testing based on our previous work, where we demonstrated how library calls may be intercepted using wrappers as well as using the kernel to separate the memory of a process into regions, based on the (statically/dynamically) linked libraries that a program uses. By monitoring function calls to libraries or the main executable, we can determine if a high-level execution signature (which depends not only on the occurrence, but also the sequence and number of calls) fits a pattern of a possible attack against a system under test. We can, then, (a) determine whether a call should go ahead, (b) determine whether the arguments are acceptable and (c) ensure that we will be informed when there is suspicion of foul play. We then demonstrate how these techniques may be used in student training sessions to explore the structure of software systems and determine how such systems respond to specific input sequences designed to trigger bugs or demonstrate unexpected behavior.
Conference Paper
Full-text available
We propose a system that allows policy to be implemented at the library call level. Under our scheme, calls to libraries are monitored and their arguments examined to ensure that they comply with the security policy associated with the running program. Our system automatically creates wrappers for libraries so that calls to external functions in the library are vectored to a policy enforcement engine. In this paper, we describe our system, which screens calls to protected functions, while allowing the implementation of a high level form of control flow integrity based on library calls. It is a transparent approach that can protect applications in many different domains and real-life environments.
Full-text available
We introduce a system that eliminates the need to run programs in privileged process contexts. Using our system, programs run unprivileged but may execute certain operations with elevated privileges as determined by a configurable policy eliminating the need for suid or sgid binaries. We present the design and analysis of the "Systrace" facility which supports fine grained process confinement, intrusion detection, auditing and privilege elevation. It also facilitates the often di#- cult process of policy generation. With Systrace, it is possible to generate policies automatically in a training session or generate them interactively during program execution. The policies describe the desired behavior of services or user applications on a system call level and are enforced to prevent operations that are not explicitly permitted. We show that Systrace is e#cient and does not impose significant performance penalties.
Control flow defenses against ROP either use strict, expensive, but strong protection against redirected RET instructions with shadow stacks, or much faster but weaker protections without. In this work we study the inherent overheads of shadow stack schemes. We find that the overhead is roughly 10% for a traditional shadow stack. We then design a new scheme, the parallel shadow stack, and show that its performance cost is significantly less: 3.5%. Our measurements suggest it will not be easy to improve performance on current x86 processors further, due to inherent costs associated with RET and memory load/store instructions. We conclude with a discussion of the design decisions in our shadow stack instrumentation, and possible lighter-weight alternatives.
Conference Paper
Having reliable security in systems is of the utmost importance. However, the existing framework of writing, distributing and linking against code in the form of libraries and/or modules does a very poor job of keeping track of who has access to what code and who can call what function. The status-quo is insufficient for a variety of reasons. As the amount of code written that represents some kind of a rights-protected entity increases, we need a systematic, easily adopted framework for designating who has access to what code, and under which conditions. While adding access controls to libraries and modules (as well as functions held securely within them), we also give regard to the performance characteristics and ease-of use considerations. In this vein, we discuss the design and implementation of a framework (called SecModule) used for generating (and using) libraries under access controls, as well as performance measurements of invoking functions that are held inside the protected library.
System call interposition allows the kernel security model to be extended. However, when combined with current operating systems, it is open to concurrency vulnerabili-ties leading to privilege escalation and audit bypass. We discuss the theory and practice of system call wrapper concurrency vulnerabilities, and demonstrate exploit tech-niques against GSWTK, Systrace, and CerbNG.
Conference Paper
Current software attacks often build on exploits that subvert machine-code execution. The enforcement of a basic safety property, Control-Flow Integrity (CFI), can prevent such attacks from arbitrarily controlling program behavior. CFI enforcement is simple, and its guarantees can be established formally even with respect to powerful adversaries. Moreover, CFI enforcement is practical: it is compatible with existing software and can be done efficiently using software rewriting in commodity systems. Finally, CFI provides a useful foundation for enforcing further security policies, as we demonstrate with efficient software implementations of a protected shadow call stack and of access control for memory regions.
Protection of computations and information is an important aspect of a computer utility. In a system which uses segmentation as a memory addressing scheme, protection can be achieved in part by associating concentric rings of decreasing access privilege with a computation. This paper describes hardware processor mechanisms for implementing these rings of protection. The mechanisms allow cross-ring calls and subsequent returns to occur without trapping to the supervisor. Automatic hardware validation of references across ring boundaries is also performed. Thus, a call by a user procedure to a protected subsystem (including the the supervisor) is identical to a call to a companion user procedure. The mechanisms of passing and referencing arguments are the same in both cases as well.
Conference Paper
Buffer overflow attack can inflict upon almost arbitrary programs and is one of the most common vulnerabilities that can seriously compromise the security of a network-attached computer system. This paper presents a compiler-based solution to the notorious buffer overflow attack problem. Using this solution, users can prevent attackers from compromising their systems by changing the return address to execute injected code, which is the most common method used in buffer overflow attacks. Return address defender (RAD) is a simple compiler patch that automatically creates a safe area to store a copy of return addresses and automatically adds protection code into applications that it compiles to defend programs against buffer overflow attacks. Using it to protect a program does not need to modify the source code of the protected programs. Moreover, RAD does not change the layout of stack frames, so binary code it generated is compatible with existing libraries and other object files. Empirical performance measurements on a fully operational RAD prototype show that programs protected by RAD only experience a factor of between 1.01 to 1.31 slow-down. In this paper we present the principle of buffer overflow attacks, a taxonomy of defense methods, the implementation details of RAD, and the performance analysis of the RAD prototype
Use of HTTP protocol by the TU-BS Sophos Repository
  • V Prevelakis
V. Prevelakis, "Use of HTTP protocol by the TU-BS Sophos Repository," TU Braunschweig, Tech. Rep., 2017.
Page Fault Exception Handler
  • D P Bovet
  • M Cesati
D. P. Bovet and M. Cesati, "Page Fault Exception Handler," in Understanding the Linux Kernel, 3rd Edition. O'Reilly, 2005, ch. 9.4.
Sophos bogus update report
  • M Tsantekidis
  • V Prevelakis
M. Tsantekidis and V. Prevelakis, "Sophos bogus update report," TU Braunschweig, Tech. Rep., 2017.
Multics data security
  • H I S Inc
H. I. S. Inc, "Multics data security," 1983. [Online]. Available:
Shining light on shadow stacks
  • N Burow
  • X Zhang
  • M Payer
N. Burow, X. Zhang, and M. Payer, "Shining light on shadow stacks," CoRR, vol. abs/1811.03165, 2018.
  • G Macmanus
  • Saelo
G. MacManus, hal, and saelo, "Nginx HTTP Server 1.3.9-1.4.0 Chunked Encoding Stack Buffer Overflow," exploit/linux/http/nginx chunked size, February 2013.