Fig 2 - uploaded by Golden G. Richard III
Content may be subject to copyright.
Overview of the Mac OS X virtual memory system, which resides inside the Mach portion of the OS X kernel. A new pager, called the compressor pager, encapsulates most of the details of page compression/decompression. Adapted from Fig. 8-6 in Singh (2006).
Source publication
The forensics community is increasingly embracing the use of memory analysis to enhance traditional storage-based forensics techniques, because memory analysis yields a wealth of information not available on non-volatile storage. Memory analysis involves capture of a system's physical memory so that the live state of a system can be investigated, i...
Contexts in source publication
Context 1
... OS X employs a hybrid kernel, containing both BSD and Mach components. The virtual memory system is implemented primarily inside the Mach portion, as illus- trated in Fig. 2. Each task (process) in Mac OS X has a flat address space represented in the kernel by a vm_map, which includes a doubly-linked list of vm_map_entry objects, accessible via the vm_map_header for the vm_map. Each of the vm_map_entry objects represents a set of contiguous memory addresses in the linear address space of the task and has ...
Context 2
... is a generic API that implements compressed swap. In particular, it manages pages that are being Fig. 3. High level structure of the compressor pager and compressor data structures, illustrating the internals of the highlighted "pager" in Fig. 2. A compressor_pager is associated with each vm_map_entry object, which tracks a set of contiguous pages in the virtual address space of a task, including the slots for compressed pages. The global compressor object stores and manages the compressed pages of all ...
Citations
... Although swap space forms the backbone of the virtual memory model and is one of the prime evidence locations, swap space remains elusive during memory forensics, and various difficulties are faced during its analysis as shown by Richard and Case [27]. This results in inconsistent memory forensic results. ...
Over the past decade, forensic investigators have incorporated memory forensics as a critical part of their investigation. Memory forensics yield substantial results that otherwise would be lost if the traditional “pull the plug” procedure is followed. However, physical memory is characterized by its dynamic and volatile nature that constantly changes the state of memory. This gives rise to challenges that reduce the reliability of physical memory forensics due to a lack of correctness, completeness, accuracy, or consistency of results. While contemporary work is focused on the acquisition or analysis of memory artifacts, little work has been done to ensure the reliability during memory forensics process. This paper presents the critical analysis of reliability in current date memory forensics and is an attempt to provide a thorough and systematic approach to ensure reliability during physical memory forensics. In the scope of this motive, the paper surveys the various memory management aspects that are crucial for reconstructing reliable memory artifacts. This includes identifying and exploring physical memory address layout, memory address space, memory address translation, memory access methods, or the low-level internal memory data structures, upon which memory forensics is reliant. The paper then identifies the prominent challenges faced during the memory forensics process and proposes a set of practices and possible suggestions to be followed or considered at different stages of physical memory forensics. These practices, if implemented, help to achieve maximum evidence reliability. Finally, a series of experiments are performed using three infamous memory acquisition tools: FTK Imager, Magnet RAM Capture, and Belkasoft RAM Capturer, on 32 GB and 16 GB Windows 10 machines to support the proposed claims. The paper also sketches the important future directions in memory forensics reliability.
... Both Linux and Mac OS compress the RAM to reduce the swapping of these files. Richard & Case (2014) developed four plug-ins that work within Volatility which is used to analyze the compressed RAM. (2) Roussev et al. (2014) presented the tool called sdkernel, which is used to identify the version of the OS kernel. ...
Publications in the digital forensics domain frequently come with tools – a small piece of functional software. These tools are often released to the public for others to reproduce results or use them for their own purposes. However, there has been no study on the tools to understand better what is available and what is missing. For this paper we analyzed almost 800 articles from pertinent venues from 2014 to 2019 to answer the following three questions (1) what tools (i.e., in which domains of digital forensics): have been released; (2) are they still available, maintained, and documented; and (3) are there possibilities to enhance the status quo? We found 62 different tools which we categorized according to digital forensics subfields. Only 33 of these tools were found to be publicly available, the majority of these were not maintained after development. In order to enhance the status quo, one recommendation is a centralized repository specifically for tested tools. This will require tool researchers (developers) to spend more time on code documentation and preferably develop plugins instead of stand-alone tools.
... The power of modern CPUs, combined with the performance of modern compression algorithms, means that a considerable number of pages can be efficiently compressed and decompressed from one physical page inside the store. Case and Richard analyzed these stores on Linux and Mac systems in a 2014 DFRWS paper (Case and Richard, 2014). Microsoft did not add support for compressed stores until Windows 10, and recently a team from FireEye presented their analysis of the Windows 10 stores. ...
... All of these reference pages that are still in memory, but that are marked as non-present in the particular address space being examined. We skip pages in the compressed state as previous work incorporated recovery of these (Case and Richard, 2014). We also skip pages on any the free queues as we do not currently believe that these pages are in a suitable state for recovery based on our reading of the kernel source code. ...
Memory forensics is the examination of volatile memory (RAM) for artifacts related to a digital investigation. Memory forensics has become mainstream in recent years because it allows recovery of a wide variety of artifacts that are never written to the file system and are therefore not available when performing traditional filesystem forensics. To analyze memory samples, an investigator can use one of several available memory analysis frameworks, which are responsible for parsing and presenting the raw data in a meaningful way. A core task of these frameworks is the discovery and reordering of non-contiguous physical pages in a memory sample into the ordered virtual address spaces used by the operating system and running processes to organize their code and data. Commonly referred to as address translation, this task requires a thorough understanding of the memory management mechanisms of the hardware architecture and operating system version of the device from which the memory sample was acquired. Given its critical role in memory analysis, there has been significant interest in studying the operating system mechanisms responsible for allocating and managing physical pages so that they can be accurately modeled by memory analysis frameworks. The more thoroughly the page handling mechanisms are modeled in memory forensics tools, the more pages can be scrutinized during memory analysis. This leads to more artifacts being reconstructed and made available to an investigator. In this paper, we present the results of our analysis of the macOS page queues subsystem. macOS tracks pages in a number of different states using a set of queues and as we will illustrate, the reconstruction of data from these queues allows a significant number of memory pages to be analyzed that are currently ignored by memory forensics tools. Through incorporation of these artifacts into analysis, memory analysis frameworks can present an even richer set of artifacts and data to investigators than ever before.
... Memory compression can significantly reduce disk I/O for swap operations and thus reduce the execution time when there are sufficient computing power and energy. Therefore, it has been used in one way or another by most commercial OSs for personal computers [24], [25]. A few compressed swap schemes for server systems have also been proposed to meet the strong memory demand of highly consolidated VMs or memory-intensive workloads [26]- [28]. ...
The limited memory capacity of mobile devices leads to the popular use of compressed swap schemes, which reduce the I/O operations involving the swapping in and out of infrequently accessed pages. However, most of the current compressed swap schemes indiscriminately compress and store all swap-out pages. Considering that both energy and computing power are scarce resources in mobile devices, and modern applications frequently deal with already-compressed multimedia data, this blind approach may cause adverse impacts. In addition, they focus only on anonymous pages and not on file-mapped pages, because the latter are backed by on-disk files. However, our observations revealed that, in mobile devices, file-mapped pages consume significantly more memory than anonymous pages. Last but not least, most of the current compressed swap schemes blindly follow the least-recently-used (LRU) discipline when choosing the victim pages for replacement, not considering the compression ratio or data density of the cached pages. To overcome the aforementioned problems and maximize the memory efficiency, we propose a compressed swap scheme, called enhanced zswap (ezswap), for mobile devices. ezswap accommodates not only anonymous pages, but also clean file-mapped pages. It estimates the compression ratio of incoming pages with their information entropy, and selectively compresses and caches the pages only with beneficial compression ratios. In addition, its admission control and cache replacement algorithms are based on a costbenefit model that considers not only the access recency of cached pages but also their information density and expected eviction cost. The proposed scheme was implemented in the Linux kernel for Android. Our evaluation with a series of commercial applications demonstrated that it reduced the amount of flash memory read by up to 55%, thereby improving the application launch time by up to 22% in comparison to the original zswap.
... OS X malware has also been on the increase [24], but there is limited published research in OS X malware analysis and detection. For example, a small number of researchers have developed OS X malware and Rootkit detection techniques, and malware detectors by tracing suspicious activities in memory (like unwanted access, read, write and execute) [25][26][27]. However, applying machine learning to detect OS X malware is limited to the Walkup approach [28], which utilized Information Gain (IG) to select effective features for supervised classification of OS X malware. ...
With the increasing market share of Mac OS X operating system, there is a corresponding increase in the number of malicious programs (malware) designed to exploit vulnerabilities on Mac OS X platforms. However, existing manual and heuristic OS X malware detection techniques are not capable of coping with such a high rate of malware. While machine learning techniques offer promising results in automated detection of Windows and Android malware, there have been limited efforts in extending them to OS X malware detection. In this paper, we propose a supervised machine learning model. The model applies kernel base Support Vector Machine and a novel weighting measure based on application library calls to detect OS X malware. For training and evaluating the model, a dataset with a combination of 152 malware and 450 benign were created. Using common supervised Machine Learning algorithm on the dataset, we obtain over 91% detection accuracy with 3.9% false alarm rate. We also utilize Synthetic Minority Over-sampling Technique (SMOTE) to create three synthetic datasets with different distributions based on the refined version of collected dataset to investigate impact of different sample sizes on accuracy of malware detection. Using SMOTE datasets we could achieve over 96% detection accuracy and false alarm of less than 4%. All malware classification experiments are tested using cross validation technique. Our results reflect that increasing sample size in synthetic datasets has direct positive effect on detection accuracy while increases false alarm rate in compare to the original dataset.
... Other projects, like (Volatility Foundation (2007e2015) and The Rekall Team, have confined themselves to a single language (Python); however, they only deal with a single (if important) sub-domain e memory analysis. Also, users must learn to work around the substantial performance deficiencies of the Python language, which can be two-three orders of magnitude slower than a C-based solution (Richard and Case, 2014). ...
The rapid growth of raw data volume requiring forensic processing has become one of the top concerns of forensic analysts. At present, there are no readily available solutions that provide: a) open and flexible integration of existing forensic tools into a processing pipeline; and b) scale-out architecture that is compatible with common cloud technologies. Containers, lightweight OS-level virtualized environments, are quickly becoming the preferred architectural unit for building large-scale data processing systems. We present a container-based software framework, SCARF, which applies this approach to forensic computations. Our prototype demonstrates its practicality by providing low-cost integration of both custom code and a variety of third-party tools via simple data interfaces. The resulting system fits well with the data parallel nature of most forensic tasks, which tend to have few dependencies that limit parallel execution. Our experimental evaluation shows that for several types of processing tasks–such as hashing, indexing and bulk processing–performance scales almost linearly with the addition of hardware resources. We show that the software engineering effort to integrate new tools is quite modest, and all the critical task scheduling and resource allocation are automatically managed by the container orchestration runtime–Docker Swarm, or similar.
... Many of these plugins have since been integrated into the stable Volatility release. Additional capabilities for Mac OS X analysis continue to be added to Volatility; for example, in 2014, Case and Richard designed Volatility plugins to address Mac OS X's compressed RAM facilities, providing automatic decompression of compressed areas of the physical memory address space (Richard and Case, 2014). ...
Abstract In the last few years there has been a sharp increase in the use of Mac OS X systems in professional settings. This has led to increased activity in the development of malware and attack toolkits focused specifically on OS X systems, and unfortunately, these increasingly powerful offensive capabilities have not (yet) resulted in better defensive research. Only a few public defensive research efforts currently exist and these only cover a portion of the attack surface that malicious OS X software has access to, particularly regarding kernel-level malware. In this paper, we present new rootkit detection techniques that attempt to close the gap between offense and defense, with a specific focus on kernel-mode components. The new detection techniques in this paper were motivated by analyzing currently available detection strategies for Windows and Linux, and noting associated deficiencies in detection schemes for Mac OS X. For each missing capability, OS X was studied to see if a similar operating system facility existed and if it could be abused by malware. For those fitting these criteria, new detection techniques were created, and these are discussed in detail in the paper. For each new rootkit detection technique we propose, a Volatility plugin was developed. Volatility is currently by far the most popular memory forensics framework in incident response and malware analysis, and by incorporating our work into Volatility, it can become immediately useful to the community. The paper concludes with an evaluation of the plugins, to illustrate their usefulness.
... Although common focus lies on compression of data on disc storage, for some usages, squeezing transient memory is also practically beneficial. For instance, the zram module of modern Linux kernels[35]compresses blocks of the main memory in order to prevent the system from running out of working memory. Compressing RAM is sometimes more preferable than storing transient data on secondary storage (e.g., in a swap file), as the latter poses a more severe performance loss. ...
For both the Lempel Ziv 77- and 78-factorization we propose algorithms
generating the respective factorization using
bits (for any positive constant ) working space (including the
space for the output) for any text of size \n\ over an integer alphabet in
time.
The first step required to perform any analysis of a physical memory image is the reconstruction of the virtual address spaces, which allows translating virtual addresses to their corresponding physical offsets. However, this phase is often overlooked and the challenges related to it are rarely discussed in the literature. Practical tools solve the problem by using a set of custom heuristics tailored on a very small number of well-known operating systems running on few architectures.
In this paper, we look for the first time at all the different ways the virtual to physical translation can be operated in 10 different CPU architectures. In each case, we study the inviolable constraints imposed by the MMU that can be used to build signatures to recover the required data structures from memory without any knowledge about the running operating system. We build a proof-of-concept tool to experiment with the extraction of virtual address spaces showing the challenges of performing an OS-agnostic virtual to physical address translation in real-world scenarios. We conduct experiments on a large set of 26 different OSs and a use case on a real hardware device. Finally, we show a possible usage of our technique to retrieve information about user space processes running on an unknown OS without any knowledge of its internals.
