Chapter

Techniques Implemented in Software Protectors: A Journey with DBI Through What Protectors Use to Detect Bad Guys

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Software protectors are products for shielding a binary executable with transformations that obfuscate and compress its original bytes in order to reveal them only during execution. In addition, they implement early-stage evasion techniques that actively look for the presence of someone who is trying to study or break such protections, since analysis environments used to this end introduce typical artifacts in the execution. In this paper we analyze a plethora of evasions used by protectors through dynamic binary instrumentation (DBI), a technique that augments the execution of a program with capabilities of monitoring and altering it up to the instruction-level granularity. As result of the analysis, we survey what artifacts are searched by the most important software protectors present on the market, dividing them according to the type of artifacts they target: environmental, analysis tools, and DBI itself .

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Complex malware samples feature measures to impede automatic and manual analyses, making their investigation cumbersome. While automatic characterization of malware benefits from recently proposed designs for passive monitoring, the subsequent dissection process still sees human analysts struggling with adversarial behaviors, many of which also closely resemble those studied for automatic systems. This gap affects the day-to-day analysis of complex samples and researchers have not yet attempted to bridge it. We make a first step down this road by proposing a design that can reconcile transparency requirements with manipulation capabilities required for dissection. Our open-source prototype BluePill (i) offers a customizable execution environment that remains stealthy when analysts intervene to alter instructions and data or run third-party tools, (ii) is extensible to counteract newly encountered anti-analysis measures using insights from the dissection, and (iii) can accommodate program analyses that aid analysts, as we explore for taint analysis. On a set of highly evasive samples BluePill resulted as stealthy as commercial sandboxes while offering new intervention and customization capabilities for dissection.
Chapter
Full-text available
Modern CPU architectures offer strong isolation guarantees towards user applications in the form of enclaves. However, Intel’s threat model for SGX assumes fully trusted enclaves and there doubt about how realistic this is. In particular, it is unclear to what extent enclave malware could harm a system. In this work, we practically demonstrate the first enclave malware which fully and stealthily impersonates its host application. Together with poorly-deployed application isolation on personal computers, such malware can not only steal or encrypt documents for extortion but also act on the user’s behalf, e.g., send phishing emails or mount denial-of-service attacks. Our SGX-ROP attack uses new TSX-based memory-disclosure primitive and a write-anything-anywhere primitive to construct a code-reuse attack from within an enclave which is then inadvertently executed by the host application. With SGX-ROP, we bypass ASLR, stack canaries, and address sanitizer. We demonstrate that instead of protecting users from harm, SGX currently poses a security threat, facilitating so-called super-malware with ready-to-hit exploits. With our results, we demystify the enclave malware threat and lay ground for future research on defenses against enclave malware.
Conference Paper
Full-text available
Dynamic binary instrumentation (DBI) techniques allow for monitoring and possibly altering the execution of a running program up to the instruction level granularity. The ease of use and flexibility of DBI primitives has made them popular in a large body of research in different domains, including software security. Lately, the suitability of DBI for security has been questioned in light of transparency concerns from artifacts that popular frameworks introduce in the execution: while they do not perturb benign programs, a dedicated adversary may detect their presence and defeat the analysis. The contributions we provide are two-fold. We first present the abstraction and inner workings of DBI frameworks, how DBI assisted prominent security research works, and alternative solutions. We then dive into the DBI evasion and escape problems, discussing attack surfaces, transparency concerns, and possible mitigations. We make available to the community a library of detection patterns and stopgap measures that could be of interest to DBI users.
Article
Full-text available
This study advances research in offensive technology by proposing return oriented programming (ROP) as a means to achieve code obfuscation. The key inspiration is that ROP's unique structure poses various challenges to malware analysis compared to traditional shellcode inspection and detection. The proposed ROP‐based attack vector provides two unique features: (i) the ability to automatically analyse and generate equivalent ROP chains for a given code, and (ii) the ability to reuse legitimate code found in an executable in the form of ROP gadgets. To this end, a software tool named ROPInjector was developed which, given any piece of shellcode and any legitimate executable file, it transforms the shellcode to its ROP equivalent re‐using the available code in the executable and finally patches the ROP chain infecting the executable. After trying various combinations of evasion techniques, the results show that ROPInjector can evade nearly and completely all antivirus software employed in the online VirusTotal service, making ROP an effective ingredient for code obfuscation. This attack vector poses a serious threat which malicious actors can take advantage to perform cyber‐attack campaigns.
Conference Paper
Full-text available
Recent years have witnessed code reuse techniques being employed to craft entire programs such as Jekyll apps, malware droppers, and persistent data-only rootkits. The increased complexity observed in such payloads calls for specific techniques and tools that can help in their analysis. In this paper we propose novel ideas for static analysis of ROP code and apply them to study prominent payloads targeting the Windows platform. Unlike state-of-the-art approaches, we do not require the ROP activation context be reproduced for the analysis. We then propose a guessing mechanism to identify gadget sources for payloads found in documents or over the network.
Conference Paper
Full-text available
In recent years, researchers have come up with proof of concepts of seemingly benign applications such as InstaStock and Jekyll that remain dormant until triggered by an attacker-crafted condition, which activates a malicious behavior, eluding code review and signing mechanisms. In this paper, we make a step forward by describing a stealthy injection vector design approach based on Return Oriented Programming (ROP) code reuse that provides two main novel features: 1) the ability to defer the specification of the malicious behavior until the attack is struck, allowing fine-grained targeting of the malware and reuse of the same infection vector for delivering multiple payloads over time; 2) the ability to conceal the ROP chain that specifies the malicious behavior to an analyst by using encryption. We argue that such an infection vector might be a dangerous weapon in the hands of advanced persistent threat actors. As an additional contribution, we report on a preliminary experimental investigation that seems to suggest that ROP-encoded malicious payloads are likely to pass unnoticed by current security solutions, making ROP an effective malware design ingredient.
Conference Paper
Full-text available
Exploits based on ROP (Return-Oriented Programming) are increasingly present in advanced attack scenarios. Testing systems for ROP-based attacks can be valuable for improving the security and reliability of software. In this paper, we propose ROPMATE, the first Visual Analytics system specifically designed to assist human red team ROP exploit builders. In contrast, previous ROP tools typically require users to inspect a puzzle of hundreds or thousands of lines of textual information, making it a daunting task. ROPMATE presents builders with a clear interface of well-defined and semantically meaningful gadgets, i.e., fragments of code already present in the binary application that can be chained to form fully-functional exploits. The system supports incrementally building exploits by suggesting gadget candidates filtered according to constraints on preserved registers and accessed memory. Several visual aids are offered to identify suitable gadgets and assemble them into semantically correct chains. We report on a preliminary user study that shows how ROPMATE can assist users in building ROP chains.
Conference Paper
Full-text available
On-stack replacement (OSR) is essential technology for adaptive optimization, allowing changes to code actively executing in a managed runtime. The engineering aspects of OSR are well-known among VM architects, with several implementations available to date. However, OSR is yet to be explored as a general means to transfer execution between related program versions, which can pave the road to unprecedented applications that stretch beyond VMs. We aim at filling this gap with a constructive and provably correct OSR framework, allowing a class of general-purpose transformation functions to yield a special-purpose replacement. We describe and evaluate an implementation of our technique in LLVM. As a novel application of OSR, we present a feasibility study on debugging of optimized code, showing how our techniques can be used to fix variables holding incorrect values at breakpoints due to optimizations.
Conference Paper
Full-text available
Automated dynamic malware analysis systems are important in combating the proliferation of modern malware. Unfortunately, malware can often easily detect and evade these systems. Competition between malware authors and analysis system developers has pushed each to continually evolve their tactics for countering the other. In this paper we systematically review i) "fingerprint"-based evasion techniques against automated dynamic malware analysis systems for PC, mobile, and web, ii) evasion detection, iii) evasion mitigation, and iv) offensive and defensive evasion case studies. We also discuss difficulties in experimental evaluation, highlight future directions in offensive and defensive research, and briefly survey related topics in anti-analysis.
Conference Paper
Full-text available
Malicious software, better known as malware, is a major threat to society. Malware today typically employ a technique called obfuscation. Obfuscation detection in malware is a well-documented problem and has been analyzed using dynamic analysis. However, many tools that detect obfuscation in malware make no attempts to use the presence of obfuscation as a method of detecting malware because their schemes would also detect benign applications. We present three main contributions. First, we conduct a unique study into the prevalence of obfuscation in benign applications. Second, we create discriminating features that can distinguish obfuscation in benign applications versus malware. Third, we prove that using the presence of obfuscation can detect previously hard-to-detect malware. Our results show that for our set of programs, we are able to reduce the number of malware missed by five market-leading AV tools by 25% while only falsely detecting 2.45% of tested benign applications.
Article
Full-text available
Vulnerabilities in low-level systems software leave users exposed to malware, theft, and monitoring. Automatic software diversity makes weaponization of these vulnerabilities much harder. The premise of software diversity is simple: randomizing program features conceals them from adversaries; however, securely and transparently implementing automatic software diversity is anything but simple.
Conference Paper
Full-text available
Malware is one of the biggest security threats on the Internet today and deploying effective defensive solutions requires the rapid analysis of a continuously increasing number of malware samples. With the proliferation of metamorphic malware the analysis is further complicated as the efficacy of signature-based static analysis systems is greatly reduced. While dynamic malware analysis is an effective alternative, the approach faces significant challenges as the ever increasing number of samples requiring analysis places a burden on hardware resources. At the same time modern malware can both detect the monitoring environment and hide in unmonitored corners of the system. In this paper we present DRAKVUF, a novel dynamic malware analysis system designed to address these challenges by building on the latest hardware virtualization extensions and the Xen hypervisor. We present a technique for improving stealth by initiating the execution of malware samples without leaving any trace in the analysis machine. We also present novel techniques to eliminate blind-spots created by kernel-mode rootkits by extending the scope of monitoring to include kernel internal functions, and to monitor file-system accesses through the kernel's heap allocations. With extensive tests performed on recent malware samples we show that DRAKVUF achieves significant improvements in conserving hardware resources while providing a stealthy, in-depth view into the behavior of modern malware.
Article
Full-text available
Malware analysis can be an efficient way to combat mali-cious code, however, miscreants are constructing heavily ar-moured samples in order to stymie the observation of their artefacts. Security practitioners make heavy use of various virtualization techniques to create sandboxing environments that provide a certain level of isolation between the host and the code being analysed. However, most of these are easy to be detected and evaded. The introduction of hardware assisted virtualization (Intel VT and AMD-V) made the cre-ation of novel, out-of-the-guest malware analysis platforms possible. These allow for a high level of transparency by re-siding completely outside the guest operating system being examined, thus conventional in-memory detection scans are ineffective. Furthermore, such analyzers resolve the short-comings that stem from inaccurate system emulation, in-guest timings, privileged operations and so on. In this paper, we introduce novel approaches that make the detection of hardware assisted virtualization platforms and out-of-the-guest malware analysis frameworks possible. To demonstrate our concepts, we implemented an applica-tion framework called nEther that is capable of detecting the out-of-the-guest malware analysis framework Ether [6].
Article
Full-text available
Process virtualization provides a virtual execution environment within which an unmodified application can be monitored and controlled while it executes. The provided layer of control can be used for purposes ranging from sandboxing to compatibility to profiling. The additional operations required for this layer are performed clandestinely alongside regular program execution. Software dynamic instrumentation is one method for implementing process virtualization which dynamically instruments an application such that the application's code and the inserted code are interleaved together. DynamoRIO is a process virtualization system implemented using software code cache techniques that allows users to build customized dynamic instrumentation tools. There are many challenges to building such a runtime system. One major obstacle is transparency. In order to support executing arbitrary applications, DynamoRIO must be fully transparent so that an application cannot distinguish between running inside the virtual environment and native execution. In addition, any desired extra operations for a particular tool must avoid interfering with the behavior of the application. Transparency has historically been provided on an ad-hoc basis, as a reaction to observed problems in target applications. This paper identifies a necessary set of transparency requirements for running mainstream Windows and Linux applications. We discuss possible solutions to each transparency issue, evaluate tradeoffs between different choices, and identify cases where maintaining transparency is not practically solvable. We believe this will provide a guideline for better design and implementation of transparent dynamic instrumentation, as well as other similar process virtualization systems using software code caches.
Conference Paper
Full-text available
Malwares attempt to evade AV scanners using various obfuscation techniques. Packing is a popular obfuscation technique used by 80% of malwares. In this paper, we propose a generic unpacking mechanism to find the original entry point (OEP) using entropy analysis. The experiment using 110 packed executables demonstrates the proposed mechanism can locate the OEPs of 72% of the packed executables. Furthermore, we show how the mechanism could be applied to packed malwares.
Conference Paper
Full-text available
Malicious programs spy on users' behavior and compromise their privacy. Even software from reputable vendors, such as Google Desktop and Sony DRM media player, may per- form undesirable actions. Unfortunately, existing techniques for detecting malware and analyzing unknown code samples are insufficient and have significant shortcomings. We ob- serve that malicious information access and processing be- havior is the fundamental trait of numerous malware cate- gories breaching users' privacy (including keyloggers, pass- word thieves, network sniffers, stealth backdoors, spyware and rootkits), which separates these malicious applications from benign software. We propose a system, Panorama, to detect and analyze malware by capturing this fundamental trait. In our extensive experiments, Panorama successfully detected all the malware samples and had very few false positives. Furthermore, by using Google Desktop as a case study, we show that our system can accurately capture its information access and processing behavior, and we can con- firm that it does send back sensitive information to remote servers in certain settings. We believe that a system such ∗This material is based upon work partially supported by the National Science Foundation under Grants No. 0311808, No. 0433540, No. 0448452, No. 0627511, and CCF-0424422. Partial support was also provided by the International Tech- nology Alliance, and by the U.S. Army Research Office under the Cyber-TA Research Grant No. W911NF-06-1-0316, and under grant DAAD19-02-1-0389 through CyLab at Carnegie Mellon. Moreover, this work was supported by the Austrian Science Foundation (FWF) under grant P18157, the FIT-IT project Pathfinder, and the Secure Business Austria compe- tence center. †Affiliated with Carnegie Mellon University and College of
Conference Paper
Full-text available
Robust and powerful software instrumentation tools are essential for program analysis tasks such as profiling, performance evaluation, and bug detection. To meet this need, we have developed a new instrumentation system called Pin. Our goals are to provide easy-to-use, portable, transparent, and efficient instrumentation. Instrumentation tools (called Pintools) are written in C/C++ using Pin's rich API. Pin follows the model of ATOM, allowing the tool writer to analyze an application at the instruction level without the need for detailed knowledge of the underlying instruction set. The API is designed to be architecture independent whenever possible, making Pintools source compatible across different architectures. However, a Pintool can access architecture-specific details when necessary. Instrumentation with Pin is mostly transparent as the application and Pintool observe the application's original, uninstrumented behavior. Pin uses dynamic compilation to instrument executables while they are running. For efficiency, Pin uses several techniques, including inlining, register re-allocation, liveness analysis, and instruction scheduling to optimize instrumentation. This fully automated approach delivers significantly better instrumentation performance than similar tools. For example, Pin is 3.3x faster than Valgrind and 2x faster than DynamoRIO for basic-block counting. To illustrate Pin's versatility, we describe two Pintools in daily use to analyze production software. Pin is publicly available for Linux platforms on four architectures: IA32 (32-bit x86), EM64T (64-bit x86), Itanium®, and ARM. In the ten months since Pin 2 was released in July 2004, there have been over 3000 downloads from its website.
Article
Full-text available
Virtual machines face significant performance challenges beyond those confronted by traditional static optimizers. First, portable program representations and dynamic language features, such as dynamic class loading, force the deferral of most optimizations until runtime, inducing runtime optimization overhead. Second, modular program representations preclude many forms of whole-program interprocedural optimization. Third, virtual machines incur additional costs for runtime services such as security guarantees and automatic memory management. To address these challenges, vendors have invested considerable resources into adaptive optimization systems in production virtual machines. Today, mainstream virtual machine implementations include substantial infrastructure for online monitoring and profiling, runtime compilation, and feedback-directed optimization. As a result, adaptive optimization has begun to mature as a widespread production-level technology. This paper surveys the evolution and current state of adaptive optimization technology in virtual machines.
Article
Full-text available
The functionality of CWSandbox, a analysis tool, used for automated dynamic malware analysis, is discussed. CWSandbox execute malware samples in a simulated environment, monitor all system calls, and automatically generates a detailed report to simplify and automate the malware analyst's task of detecting malware. CWSandbox analyze a single malware execution at a time using API functions. During execution of function by a application, application is rerouted to a different location containing customized codes. API functions then performs its own operations and transfer control back to the original API function or prevents its execution. CWSandbox use in-line overwriting technique to enable API functions to intercept system calls from potentially malicious user application.
Article
Full-text available
Future high-performance virtual machines will improve performance through sophisticated online feedback-directed optimizations. This paper presents the architecture of the Jalapeño Adaptive Optimization System, a system to support leading-edge virtual machine technology and enable ongoing research on online feedback-directed optimizations. We describe the extensible system architecture, based on a federation of threads with asynchronous communication. We present an implementation of the general architecture that supports adaptive multi-level optimization based purely on statistical sampling. We empirically demonstrate that this profiling technique has low overhead and can improve startup and steady-state performance, even without the presence of online feedback-directed optimizations. The paper also describes and evaluates an online feedback-directed inlining optimization based on statistical edge sampling. The system is written completely in Java, applying the described techniques n...
Chapter
Software protectors aim to shield executable file against reversing and cracking, by implementing sophisticated mechanisms capable of hiding real binary code and by inserting, inside an executable file, pieces of control code created to reconstruct software to its original state only during execution. This process of hiding and restoring protected bytes is composed by complex code fragments which perform all essential operations, keeping in general a constant structure for all protected software. The goal of this paper is to enhance security of these types of software against crackers, using Hardware Transactional Memories (HTM) by exploiting their features in a way for which they were not designed for. The proposed methodology offers a good level of defence of the protector core part by implementing new specific checks which use hardware processor features, ensuring reliability and a good level of performance with respect to different software implementations and which can be inserted inside a just present chain of checks, enhancing whole programs protection.
Article
The cyber world is plagued with ever-evolving malware that readily infiltrate all defense mechanisms, operate viciously unbeknownst to the user, and surreptitiously exfiltrate sensitive data. Understanding the inner workings of such malware provides a leverage to effectively combat them. This understanding is pursued often through dynamic analysis which is conducted manually or automatically. Malware authors accordingly, have devised and advanced evasion techniques to thwart or evade these analyses. In this article, we present a comprehensive survey on malware dynamic analysis evasion techniques. In addition, we propose a detailed classification of these techniques and further demonstrate how their efficacy holds against different types of detection and analysis approaches. Our observations attest that evasive behavior is mostly concerned with detecting and evading sandboxes. The primary tactic of such malware we argue is fingerprinting followed by new trends for reverse Turing test tactic which aims at detecting human interaction. Furthermore, we will posit that the current defensive strategies, beginning with reactive methods to endeavors for more transparent analysis systems, are readily foiled by zero-day fingerprinting techniques or other evasion tactics such as stalling. Accordingly, we would recommend the pursuit of more generic defensive strategies with an emphasis on path exploration techniques that has the potential to thwart all the evasive tactics.
Conference Paper
Code virtualization is a highly sophisticated obfuscation technique adopted by malware authors to stay under the radar. However, the increasing complexity of code virtualization also becomes a "double-edged sword" for practical application. Due to its performance limitations and compatibility problems, code virtualization is seldom used on an entire program. Rather, it is mainly used only to safeguard the key parts of code such as security checks and encryption keys. Many techniques have been proposed to reverse engineer the virtualized code, but they share some common limitations. They assume the scope of virtualized code is known in advance and mainly focus on the classic structure of code emulator. Also, few work verifies the correctness of their deobfuscation results. In this paper, with fewer assumptions on the type and scope of code virtualization, we present a verifiable method to address the challenge of partially-virtualized binary code simplification. Our key insight is that code virtualization is a kind of process-level virtual machine (VM), and the context switch patterns when entering and exiting the VM can be used to detect the VM boundaries. Based on the scope of VM boundary, we simplify the virtualized code. We first ignore all the instructions in a given virtualized snippet that do not affect the final result of that snippet. To better revert the data obfuscation effect that encodes a variable through bitwise operations, we then run a new symbolic execution called multiple granularity symbolic execution to further simplify the trace snippet. The generated concise symbolic formulas facilitate the correctness testing of our simplification results. We have implemented our idea as an open source tool, VMHunt, and evaluated it with real-world applications and malware. The encouraging experimental results demonstrate that VMHunt is a significant improvement over the state of the art.
Chapter
Binary instrumentation is a robust and powerful technique which facilitates binary code modification of computer programs even when no source code is available. This is achieved either statically by rewriting the binary instructions of the program and then executing the altered program or dynamically, by changing the code at run-time right before it is executed. The design of most Dynamic Binary Instrumentation (DBI) frameworks puts emphasis on ease-of-use, portability, and efficiency, offering the possibility to execute inspecting analysis code from an interpositioned perspective maintaining full access to the instrumented program. This has established DBI as a powerful tool utilized for analysis tasks such as profiling, performance evaluation, and prototyping.
Chapter
Protecting a digital asset once it leaves the cyber trust boundary of its creator is a challenging security problem. The creator is an entity which can range from a single person to an entire organization. The trust boundary of an entity is represented by all the (virtual or physical) machines controlled by that entity. Digital assets range from media content to code and include items such as: music, movies, computer games, and premium software features. The business model of the creator implies sending digital assets to end-users—such that they can be consumed—in exchange for some form of compensation. A security threat in this context is represented by malicious end-users, who attack the confidentiality or integrity of digital assets, in detriment to digital asset creators and/or other end-users. Software obfuscation transformations have been proposed to protect digital assets against malicious end-users, also called Man-At-The-End (MATE) attackers. Obfuscation transforms a program into a functionally equivalent program which is harder for MATE to attack. However, obfuscation can be use both for benign and malicious purposes. Malware developers rely on obfuscation techniques to circumvent detection mechanisms and to prevent malware analysts from understanding the logic implemented by the malware. This chapter presents a tutorial of the most popular existing software obfuscation transformations and mentions published attacks against each transformation. We present a snapshot of the field of software obfuscation and indicate possible directions, which require more research.
Conference Paper
Virtualization has become an indispensable technique for scaling up the analysis of malicious code, such as for malware analysis or shellcode detection systems. Frameworks like Ether, ShellOS and an ever-increasing number of commercially-operated malware sandboxes rely on hardware-assisted virtualization. A core technology is Intel’s VT-x, which — compared to software-emulated virtulization — is believed to be stealthier, especially against evasive attackers that aim to detect virtualized systems to hide the malicious behavior of their code. We propose and evaluate low-level timing-based mechanisms to detect hardware-virtualized systems. We build upon the observation that an adversary can invoke hypervisors and trigger context switches that are noticeable both in timing and in their side effects on caching. We have locally trained and then tested our detection methodology on a wide variety of systems, including 240 PlanetLab nodes, showing a high detection accuracy. As a real-world evaluation, we detected the virtualization technology of more than 30 malware sandboxes. Finally, we demonstrate how an adversary may even use these detections to evade multi-path exploration systems that aim to explore the full behavior of a program. Our results show that VT-x is not sufficiently stealthy for reliable analysis of malicious code.
Article
Traditional malware analysis relies on virtualization or emulation technology to run samples in a confined environment, and to analyze malicious activities by instrumenting code execution. However, virtual machines and emulators inevitably create artifacts in the execution environment, making these approaches vulnerable to detection or subversion. In this paper, we present MALT, a debugging framework that employs System Management Mode, a CPU mode in the x86 architecture, to transparently study armored malware. MALT does not depend on virtualization or emulation and thus is immune to threats targeting such environments. Our approach reduces the attack surface at the software level, and advances state-of-the-art debugging transparency. MALT embodies various debugging functions, including register/memory accesses, breakpoints, and seven stepping modes. Additionally, MALT restores the system to a clean state after a debugging session. We implemented a prototype of MALT on two physical machines, and we conducted experiments by testing an array of existing anti-virtualization, anti-emulation, and packing techniques against MALT. The experimental results show that our prototype remains transparent and undetected against the samples. Furthermore, debugging and restoration introduce moderate but manageable overheads on both Windows and Linux platforms.
Conference Paper
A critical challenge when combating malware threat is how to efficiently and effectively identify the targeted victim’s environment, given an unknown malware sample. Unfortunately, existing malware analysis techniques either use a limited, fixed set of analysis environments (not effective) or employ expensive, time-consuming multi-path exploration (not efficient), making them not well-suited to solve this challenge. As such, this paper proposes a new dynamic analysis scheme to deal with this problem by applying the concept of speculative execution in this new context. Specifically, by providing multiple dynamically created, parallel, and virtual environment spaces, we speculatively execute a malware sample and adaptively switch to the right environment during the analysis. Interestingly, while our approach appears to trade space for speed, we show that it can actually use less memory space and achieve much higher speed than existing schemes. We have implemented a prototype system, GoldenEye, and evaluated it with a large real-world malware dataset. The experimental results show that GoldenEye outperforms existing solutions and can effectively and efficiently expose malware’s targeted environment, thereby speeding up the analysis in the critical battle against the emerging targeted malware threat.
Article
Run-time packers are often used by malware-writers to obfuscate their code and hinder static analysis. The packer problem has been widely studied, and several solutions have been proposed in order to generically unpack protected binaries. Nevertheless, these solutions commonly rely on a number of assumptions that may not necessarily reflect the reality of the packers used in the wild. Moreover, previous solutions fail to provide useful information about the structure of the packer or its complexity. In this paper, we describe a framework for packer analysis and we propose a taxonomy to measure the runtime complexity of packers. We evaluated our dynamic analysis system on two datasets, composed of both off-the-shelf packers and custom packed binaries. Based on the results of our experiments, we present several statistics about the packers complexity and their evolution over time.
Article
The ability to trap the execution of a binary program at desired instructions is essential in many security scenarios such as malware analysis and attack provenance. However, an increasing percent of both malicious and legitimate programs are equipped with anti-debugging and anti-instrumentation techniques, which render existing debuggers and instrumentation tools inadequate. In this paper, we present Spider, a stealthy program instrumentation framework which enables transparent, efficient and flexible instruction-level trapping based on hardware virtualization. Spider uses invisible breakpoint, a novel primitive we develop that inherits the efficiency and flexibility of software breakpoint, and utilizes hardware virtualization to hide its side-effects from the guest. We have implemented a prototype of Spider on KVM. Our evaluation shows that Spider succeeds in remaining transparent against state-of-the-art anti-debugging and anti-instrumentation techniques; the overhead of invisible breakpoint is comparable with traditional hardware breakpoint. We also demonstrate Spider's usage in various security applications.
Conference Paper
In recent years, an ever-increasing number of IT security incidents have been observed, often involving malicious software. In order to cope with the threat posed, it is essential to have a structured analysis workflow for assessment and mitigation. In this paper, we give a thorough explanation of the malware analysis workflow specified and employed by our team of analysts. It was deducted from observed work patterns and best practices with a strong focus on enabling collaboration, i.e. analyses conducted by multiple analysts in parallel in order to achieve a speed-up. The proposed workflow starts at the point where one or more malware samples have already been extracted. It consists of four phases as a whole, each with its own goals, constraints, and abort conditions. The first phase aims at gaining an overview of the current situation and specifying goals of the analysis and their respective priorities. The second phase features a preliminary analysis used to sharpen the picture of the threat, using methods of Open Source Intelligence (OSINT) and automated tools in order to obtain a quick assessment enabling first mitigation. In addition, one objective is to facilitate and prepare a more granular dissection of the malware sample, e.g. by unpacking and deobfuscation. The third phase comprises an in-depth analysis relying heavily on reverse engineering of selected parts of the malware. The selection may be influenced by earlier findings or focus on prominent aspects like nesting, functionality, or communication protocols. The final phase builds upon the results of the preceding phases, leading to tailored mitigation concepts for the specimen analysed. For each of the proposed phases, we give an overview of potential key tools, e.g. helping to gain information or improve collaboration. On a higher level, we highlight challenges to cooperative analysis and our approach to handle them. In this regard, the workflow contains adoptions of principles known from agile soft- are development methodologies. For example, Scrum is used for management of tasks and coordination, aiding the creation of a reproducible and reliable chain of results.
Article
Anti-virus vendors are confronted with a multitude of potentially malicious samples today. Receiving thousands of new samples every day is not uncommon. The signatures that detect confirmed malicious threats are mainly still created manually, so it is important to discriminate between samples that pose a new unknown threat and those that are mere variants of known malware. This survey article provides an overview of techniques based on dynamic analysis that are used to analyze potentially malicious samples. It also covers analysis programs that leverage these It also covers analysis programs that employ these techniques to assist human analysts in assessing, in a timely and appropriate manner, whether a given sample deserves closer manual inspection due to its unknown malicious behavior.
Article
Dynamic binary instrumentation systems have become popular frameworks for building custom program analysis tools. For example, Pin [8], Valgrind [9], and DynamoRIO [5] have been used to build a variety of memory checking, thread checking, cache simulation, and profiling tools. However, there has been little emphasis on building custom debugging tools with these frameworks. We introduce PinADX, a set of debugging extensions to the Pin instrumentation framework that allow easy development of custom debugger features on both Linux and Windows. This paper presents the PinADX API, demonstrates its use in a simple debugger extension, discusses the implementation details in the Pin instrumentation system, and shows that the performance is better than a native debugger for some typical debugging tasks. We believe that dynamic instrumentation technology is a powerful tool for building advanced debugging features and that an easy to use API will greatly facilitate the development of such features.
Article
Packing (or executable compression) is considered as one of the most effective anti-reverse engineering methods in the Microsoft Windows environment. Even though many reversing attacks are widely conducted in the Linux-based embedded system there is no widely used secure binary code packing tools for Linux. This paper presents two secure packing methods that use AES encryption and the UPX packer to protect the intellectual property (IP) of software from reverse engineering attacks on Linux-based embedded system. We call these methods: secure UPX and AES-encryption packing. Since the original UPX system is designed not for software protection but for code compression, we present two anti-debugging methods in the unpacking module of the secure UPX to detect or abort reverse engineering attacks. Furthermore, since embedded systems are highly resource constrained, minimizing unpacking overhead is important. Therefore, we analyze the performance of the two packing methods from the perspective of: (i) code size, (ii) execution time, and (iii) power consumption. Our analysis results show that the Secure UPX performs better than AES-encryption packing in terms of the code size, execution time, and power consumption.
Article
Nearly every malware sample is sheathed in an executable protection which must be removed before static analyses can proceed. Existing research has studied automati-cally unpacking certain protections, but has not yet caught up with many modern techniques. Contrary to prior assumptions, protected programs do not always have the property that they are reverted to a fully unprotected state at some point during the course of their execution. This work provides a novel technique for circumventing one of the most problematic features of modern software protections, so-called virtualization obfuscation. The technique enables analysis of heretofore impenetrable malware.
Conference Paper
We present new techniques that allow a return-into-libc attack to be mounted on x86 exe- cutables that calls no functions at all. Our attack combines a large number of short instruction sequences to build gadgets that allow arbitrary computation. We show how to discover such instruction sequences by means of static analysis. We make use, in an essential way, of the properties of the x86 instruction set.
Conference Paper
Malware has become the centerpiece of most security threats on the Internet. Malware analysis is an essential technology that extracts the runtime behavior of malware, and supplies signatures to detection systems and provides evidence for re- covery and cleanup. The focal point in the malware analysis battle is how to detect versus how to hide a malware ana- lyzer from malware during runtime. State-of-the-art analyz- ers reside in or emulate part of the guest operating system and its underlying hardware, making them easy to detect and evade. In this paper, we propose a transparent and ex- ternal approach to malware analysis, which is motivated by the intuition that for a malware analyzer to be transparent, it must not induce any side-effects that are unconditionally detectable by malware. Our analyzer, Ether, is based on a novel application of hardware virtualization extensions such as Intel VT, and resides completely outside of the target OS environment. Thus, there are no in-guest software compo- nents vulnerable to detection, and there are no shortcomings that arise from incomplete or inaccurate system emulation. Our experiments are based on our study of obfuscation tech- niques used to create 25,000 recent malware samples. The results show that Ether remains transparent and defeats the obfuscation tools that evade existing approaches.
Conference Paper
Dynamic binary instrumentation (DBI) frameworks make it easy to build dynamic binary analysis (DBA) tools such as checkers and profilers. Much of the focus on DBI frameworks has been on performance; little attention has been paid to their capabilities. As a result, we believe the potential of DBI has not been fully exploited. In this paper we describe Valgrind, a DBI framework designed for building heavyweight DBA tools. We focus on its unique support for shadow values-a powerful but previously little-studied and difficult-to-implement DBA technique, which requires a tool to shadow every register and memory value with another value that describes it. This support accounts for several crucial design features that distinguish Valgrind from other DBI frameworks. Because of these features, lightweight tools built with Valgrind run comparatively slowly, but Valgrind can be used to build more interesting, heavyweight tools that are difficult or impossible to build with other DBI frameworks such as Pin and DynamoRIO.
Codisasm: medium scale concatic disassembly of self-modifying binaries with overlapping instructions
  • G Bonfante
  • J Fernandez
  • J.-Y Marion
  • B Rouxel
  • F Sabatier
  • A Thierry
Defeating the transparency features of dynamic binary instrumentation
  • X Li
  • K Li
X-force: force-executing binary programs for security applications
  • F Peng
  • Z Deng
  • X Zhang
  • D Xu
  • Z Lin
  • Z Zhendong
Dynamic binary instrumentation frameworks: I know you’re there spying on me
  • F Falcón
  • N Riva
The ROP needle: hiding trigger-based injection vectors via code reuse
  • P Borrello
  • E Coppa
  • D C Elia
  • C Demetrescu
The geometry of innocent flesh on the bone: return-into-LIBC without function calls (on the x86)
  • H Shacham
Active detection and escape of dynamic binary instrumentation
  • K Sun
  • X Li
  • Y Ou
PwIN - Pwning Intel piN: why DBI is unsuitable for security applications
  • J Kirsch
  • Z Zhechev
  • B Bierbaumer
  • T Kittel