Available via license: CC BY
Content may be subject to copyright.
A preview of the PDF is not available
Dynamic analysis of virtualization- or dispatching-obfuscated applications
Abstract
Obfuscation algorithms are now widely used to prevent software reverse engineering. Binary code virtualization is one of the most powerful obfuscations technics. Another obfuscation method known as “dispatching” can be used to transform application control flow similarly to virtual machine insertion. Our research was aimed at reconstruction of control flow graph in case of both code virtualization and dispatching. To achieve this goal, we implemented de-obfuscation tool which keeps track of virtual program counter used by virtual machine emulator and reconstructs the application control flow. This paper describes experimental results of test application de-obfuscation via dynamic analysis. Both obfuscating and de-obfuscating tools were independently developed by two different teams of ISP RAS – the LLVM-based obfuscating compiler and the software environment for dynamic analysis of binary code. The paper briefly introduces both software tools and then describes results of experimental research on recovering of control flow graph of obfuscated application. Application was initially protected by specialized obfuscating LLVM-based compiler. Next, TrEx environment was used to analyze program execution trace, to find the dispatcher-protected part of application and to recover its control flow. Additionally, some software code complexity metrics for test applications were calculated to estimate obfuscation resilience provided by different versions of obfuscating compiler.
The paper describes the obfuscating transformations, which were implemented while developing an LLVM-based obfuscating compiler in ISP RAS. The proposed transformations are based on well-known obfuscation algorithms and are specifically improved to resist better to static analysis deobfuscation techniques. The application performance decrease estimation and the increase of application memory consumption estimation are presented. Also, the possibility of source code information recovery is estimated. The implemented obfuscating transformations can be applied together to the given application to provide the strong protection from the static analysis deobfuscation attacks.
ResearchGate has not been able to resolve any references for this publication.