Ryan D. Maggio’s research while affiliated with Louisiana State University and other places

The value of memory analysis during digital forensics, incident response, and malware investigations has been realized for over a decade. The power of memory forensics is based on the fact that volatile memory contains a substantial number of artifacts that are simply never recorded to disk or sent across the network in plaintext form. Orderly recovery of this data, known as structured analysis, allows for recovery of the full system state at the time of acquisition. For structured analysis to be successful, a memory analysis framework must have an accurate model of the data structures and algorithms of the target operating system and applications. Unfortunately, acquiring this layout is often a difficult task for even one version of an executable module, and the problem is only compounded when support for a wide variety of versions is desired. This issue can be manifested in several ways, including forensics frameworks being unable to process memory samples containing unsupported versions of executable code or worse, generating erroneous or incomplete results. Given the vital role memory analysis plays in modern investigations, these issues are unacceptable. In this paper, we present Seance, a system that implements automated binary analysis to provide accurate data structure layout information for different versions of targeted executed modules. The results of Seance can be consumed by analysis frameworks to accurately support all versions of a target module.

Advances in malware development have led to the widespread use of attacker toolkits that do not leave any trace in the local filesystem. This negatively impacts traditional investigative procedures that rely on filesystem analysis to reconstruct attacker activities. As a solution, memory forensics has replaced filesystem analysis in these scenarios. Unfortunately, existing memory forensics tools leave many capabilities inaccessible to all but the most experienced investigators, who are well versed in operating systems internals and reverse engineering. The goal of the research described in this paper is to make investigation of one of the greatest threats that organizations face, userland keyloggers, less error-prone and less dependent on manual reverse engineering. To accomplish this, we have added significant new capabilities to HookTracer, which is an engine capable of emulating code discovered in a physical memory captures and recording all actions taken by the emulated code. Based on this work, we present new memory forensics capabilities, embodied in a new Volatility plugin, hooktracer_messagehooks, that uses Hooktracer to automatically decide whether a hook in memory is associated with a malicious keylogger or benign software. We also include a detailed case study that illustrates our technique’s ability to successfully analyze very sophisticated keyloggers, such as Turla.

Memory forensics is the examination of volatile memory (RAM) for artifacts related to a digital investigation. Memory forensics has become mainstream in recent years because it allows recovery of a wide variety of artifacts that are never written to the file system and are therefore not available when performing traditional filesystem forensics. To analyze memory samples, an investigator can use one of several available memory analysis frameworks, which are responsible for parsing and presenting the raw data in a meaningful way. A core task of these frameworks is the discovery and reordering of non-contiguous physical pages in a memory sample into the ordered virtual address spaces used by the operating system and running processes to organize their code and data. Commonly referred to as address translation, this task requires a thorough understanding of the memory management mechanisms of the hardware architecture and operating system version of the device from which the memory sample was acquired. Given its critical role in memory analysis, there has been significant interest in studying the operating system mechanisms responsible for allocating and managing physical pages so that they can be accurately modeled by memory analysis frameworks. The more thoroughly the page handling mechanisms are modeled in memory forensics tools, the more pages can be scrutinized during memory analysis. This leads to more artifacts being reconstructed and made available to an investigator. In this paper, we present the results of our analysis of the macOS page queues subsystem. macOS tracks pages in a number of different states using a set of queues and as we will illustrate, the reconstruction of data from these queues allows a significant number of memory pages to be analyzed that are currently ignored by memory forensics tools. Through incorporation of these artifacts into analysis, memory analysis frameworks can present an even richer set of artifacts and data to investigators than ever before.

The use of memory forensics is becoming commonplace in digital investigation and incident response, as it provides critically important capabilities for detecting sophisticated malware attacks, including memory-only malware components. In this paper, we concentrate on improving analysis of API hooks, a technique commonly employed by malware to hijack the execution flow of legitimate functions. These hooks allow the malware to gain control at critical times and to exercise complete control over function arguments and return values. Existing techniques for detecting hooks, such the Volatility plugin apihooks, do a credible job, but generate numerous false positives related to non-malicious use of API hooking. Furthermore, deeper analysis to determine the nature of hooks detected by apihooks typically requires substantial skill in reverse engineering and an extensive knowledge of operating systems internals. In this paper, we present a new, highly configurable tool called hooktracer, which eliminates false positives, provides valuable insight into the operation of detected hooks, and generates portable signatures called hook traces, which can be used to rapidly investigate large numbers of machines for signs of malware infection.