Content uploaded by Marco Spaziani Brunella
Author content
All content in this area was uploaded by Marco Spaziani Brunella on Jun 27, 2019
Content may be subject to copyright.
Foreshadow-VMM: Feasibility and Network
Perspective
Marco Spaziani Brunella∗† , Giuseppe Bianchi∗† , Sara Turco†, Francesco Quaglia†, Nicola Blefari-Melazzi∗†
∗Consorzio Nazionale Interuniversitario per le Telecomunicazioni
†University of Rome Tor Vergata
{name.surname}@uniroma2.it
Abstract—On August 14, 2018, a new set of vulnerabilities
collectively named ”L1 terminal fault” were announced. Systems
with microprocessors utilizing out-of-order execution could allow
unauthorized disclosure of information residing in the L1 data
cache, by tweaking the virtual memory abstraction. The vulnera-
bility was therein mentioned for three different scenarios. In this
demo-paper, we provide practical evidence about the feasibility
of the most complex ”VMM” case of an attacker residing in a
Virtual Machine (VM), and targeting information leakage from
the host OS and other independent VMs.
I. INTRODUCTION
Network virtualization technologies hold the promise to
bring an unprecedented dynamism in configuring and deploy-
ing network services. Indeed, network virtualization enables
the creation of multiple isolated logical per-application net-
works (the so-called network ”slices” in the emerging 5G con-
text), possibly characterized by very different service delivery
models and dedicated resources, over the same physical infras-
tructure. Virtualization makes it easy to manage and develop
network services, since they are implemented as software,
completely decoupled from their supporting hardware.
However, network virtualization comes along with a sig-
nificantly wider exposure to threats and attacks. In this paper,
we are specifically concerned with a new wave of transient
execution attacks [1], which appear to be fundamental threats
to modern processor architectures extensively used in cloud-
based systems, and which may thus play havoc with the desire
of virtual infrastructure operators to rely on virtualization for
network slicing’s isolation and segregation.
Such attacks leverage access to shared memory caches
as (timing) side-channels for information leakage, and ex-
ploit hard to disable performance acceleration and com-
plex optimization techniques in modern microprocessor ar-
chitectures, such as extensive caching, branch prediction and
speculative/out-of-order executions. Indeed, the Spectre [2]
and Meltdown [3] attacks, conceived a couple of years ago,
recently evolved into novel forms of attacks, not only with the
Foreshadow vulnerability [4] addressed hereafter, but also with
even more recent attacks disclosed while this paper was being
prepared [1], and affecting multiple microprocessor brands
(Intel, ARM, AMD).
The Foreshadow attack. The Foreshadow vulnerability [4]
(a.k.a. L1 Terminal Fault - L1TF), disclosed to the public
on August 14, 2018, is an out-of-order execution attack that
Physical Core
L1 Cache
L2 Cache
Victim’s memory Attacker’s memory
Victim
VM
Logical Core 2
Attacker
VM
Logical Core 1
Cloud Server
Fig. 1. Foreshadow-VMM attack overview
provides the possibility to completely bypass the virtual mem-
ory abstraction, thus providing means to perform unauthorized
memory accesses to the L1-data cache. This vulnerability
arises by the fact that, for performance reasons, processors use
speculative execution during the virtual-to-physical memory
address translation. In particular, while the correspondence
between virtual and physical addresses is searched in the page
table (i.e., during a page table walk), the processor accesses
in parallel the L1 data cache. If the logical address has not
a mapping to the physical location, the translation process is
aborted and a terminal fault rises. However, there is a time
period before the retrieve operation in which data are still
passed to the cache, even if an access violation occurs. In
this scenario, side-channel methods may be exploited to gather
access to protected information.
Our specific interest here is on the Foreshadow-VMM
[5] variant of the L1TF attacks, as it threatens virtualized
environments (thus including virtualized network scenarios).
As long as a malicious Virtual Machine (VM) shares a same
physical core (and thus the same L1 cache) with a victim VM,
Foreshadow-VMM implies that it is possible for the attacket
to access arbitrary data L1-cached by the victim VM - Fig. 2.
Our contribution. To the best of our knowledge, unlike
e.g., the case of Foreshadow-SGX, the practicality of the
Foreshadow-VMM L1TF attack was not yet shown to date.
Indeed, while trying to reproduce it, we needed to go through
a number of technical steps, which we have documented in this
demo. We specifically worked on the page table management
in Linux, by coding a faulty process and an offending kernel
module which implement methods for triggering the terminal
fault, for navigating the page table, and for changing the
page table entry with any arbitrarily chosen physical memory
address. After triggering the fault, we have constructed a
cache probe, that continuously scans the cache for active lines978-1-5386-9376-6/19/$31.00 c
2019 IEEE
Linear Address
Oset within
Process PGD
Oset within PUD
Page Frame
Oset within PMD
Page Frame
Oset within PTE
Page Frame
Oset within
Data Frame
pgd_oset()
pud_oset()
pmd_oset()
pte_oset()
pgd_index()
mm_struct->pgd
Only 1 pgd_t
Page Frame
pud_t Page
Frame
pmd_t Page
Frame
pte_t Page
Frame
Page Frame with
User Data
Other unrelated
PUD Page Frame
Other unrelated
PMD Page Frame
Other unrelated
PTE Page Frame
Other unrelated
DataFrame
pgd_t
pud_t
pmd_t
pte_t
Fig. 2. Linux Page Table Managment (up to kernel 4.11-rc2)
and flushed them, thus performing a Flush+Reload [6] data
retrieval.
II. BACKGROU ND
A. Intel address translation
The L1TF attack leverages fundamental performance opti-
mization used by Intel processors during the virtual-to-physical
memory address translation.
We recall that Virtual memory exposes to the pro-
grammer a linear address space, divided in chunks called
pages, which is then mapped to the physical mem-
ory using Page Table (PT), which contains all pairs of
hvirtual address, phy sical addressi. For performance rea-
sons, Intel processors maintain a cache of recently translated
addresses, the Translation Lookaside Buffer (TLB), and per-
form a page walk, i,.e. a linear search throughout the page
tables, only when the required translation is not in the TLB.
In virtualized environments, a further translation step called
Extended Page Table (EPT) walk is mandated. In fact, guest
VMs run on top of the host machine’s hypervisor. Thus,
the guest VM’s physical address, obtained from the virtual
machine’s page walk, is still a virtual address for the host
machine, which has to be further translated into a physical
address. If one of these steps reports an access violation, a
page fault rises and the translation is aborted.
To speed up memory access, Intel implements some in-
parallel micro-architectural operations, adopting caches and
optimization techniques. At the core of the Foreshadow-VMM
attack, there is the fact that the first step of the translation
is done in parallel with the L1-D cache lookup, which is
referenced with the virtual address. Additionally, out-of-order
execution, is used to improve performance, by executing in-
structions ahead in the instruction flow, processing instructions
in an order that is different from how the program is written,
and avoiding to stall the CPU if some data hazard occurs.
These operations allow instructions to speculatively compute
on unauthorized memory contents even in case of access
violation, since data coming from the L1 cache are directly
passed to the out-of-order execution.
In Foreshadow-VMM the guest virtual machine has control
over its own virtual-to-physical address translation that goes
directly inside the L1 cache. When a malicious VM triggers a
terminal fault, it blocks the EPT translation but passes the guest
physical address to the cache. This allows the VM to read any
cached physical memory location (e.g., others VMs data), since
the data coming from the cache is directly passed to out-of-
order execution. When the CPU is blocked due to the terminal
fault, some transient instructions may have already fetched
some data from the cache. Moreover, as we will demonstrate
below, an untrusted guest VM can be able to point to arbitrary
physical memory by ”simply” modifying the page table entry
of its own page table.
B. Page table management in Linux
In Intel’s x64 architecture [7], Linux stores processes’s
page tables[8] in a tree-based data structure that is specific
of a single process running on the machine. The tree has four
different levels: Page Global Directory (PGD), Page Upper
Directory (PUD), Page Middle Directory (PMD) and Page
Table Entries (PTE).
The translation starts with the task struct of the process,
from which we extract the mm struct. The PGD entry is
extracted from the mm struct, wich is the root of the tree.
We can use macros such as pgd offset(),pud offset(),
pmd offset() and pte offset map() to navigate the tree and
arrive to a leaf, namely the Page Table Entry associated with
the address targeted in the translation.
Page table entries are composed of many fixed fields, which
contain the physical translation alongside with other control
bits. The bit #0 represents the present (P) flag: when it is set,
the page is in physical memory and the translation can be
performed. On the other hand, when the present flag is set to
0, the page table entry does not contain any useful data. In this
case, if a process tries to access this translation, it generates a
page fault. The state of this flag is maintained by the kernel,
thus motivating the use of a kernel module to modify a PTE,
as explained in later sections.
III. ATTACK :TECHNICAL STEPS
We now describe our approach for triggering a terminal
fault, and allow transient data transfer inside the L1D cache
from a victim VM to the attacker VM, sharing the same
physical core.
A. Setup & Overview
In order to be sure that the two machines are running on
the same physical core, two VMs have been instantiated using
Kernel-Based Virtual Machine (KVM), setting CPU affinities
to force KVM to run the two VMs on the same core. We then
provided an attacker VM with the following two tools:
•an Offending Kernel Module (OKM), triggering the
terminal fault on the Probe.
•aProbe, a user space program devised to be offended
by the OKM and to probe the cache for active lines.
At first, Probe is started. The kernel associates with such
process with a Process ID (PID) and creates the relevant page
table tree inside the kernel structure, namely the mm data
structure. The Probe instantiates a temporary variable called
tmp, which has a certain virtual address, thus creating a Page
Table Entry for that variable.
The Probe will pass it’s PID to the OKM using a character
device, passing also the virtual address of the tmp variable
in the u64 format, needed by the Offending Kernel Mo1dule
(OKM) to actually tamper the page table entry associated with
the tmp variable.
B. Page Walk
At this point, the OKM can be loaded inside the attackers’
VM and recives the PID of the Probe and the vaddr of the
tmp variable. Starting from the PID, the OKM extracts the
mm struct from the task struct, which is process-specific. With
such information, the OKM can start a page walk, as depicted
in Figure 2, to arrive to the leaf node of the tree, which contains
the Page Table Entry (PTE) for the tmp variable of the Probe,
which contains the translation between the virtual address and
the physical address of the variable, aside with other control
bits, such as the present bit of the PTE.
C. Clearing the present bit & tampering the translation
At this point, we are able to manipulate the page table
entry, setting/clearing control bits and altering the translation
between virtual and physical address. In particular, clearing
the present bit on the tmp variable PTE will cause a terminal
fault when the Probe will access again the tmp variable.
It is also possible now to tamper the translation, since the
physical address is encoded inside the PTE from bit 47 down
to 12. In fact, substituting this address with a malicious one,
allows us to read any data residing in the L1D cache on the
subsequent step.
D. Triggering the terminal fault & retrieving data
When the Probe will access again the tmp variable, the
kernel will perform a page walk, arriving on the tampered
PTE. It will find that the PTE is not valid, since the present
bit has been cleared, rising an exception which will abruptly
terminates the Probe after a certain time window. In the
meanwhile, the access to the tmp variable has been passed
to out-of-order execution with the tampered address, making
a side effect on the cache based on data contained in such a
location, which may belong to the Host OS or to the Victim
VM. At this point, the Probe intercepts the exception and starts
to read speculatively the content of the L1-data cache, as in
[6]. At this point the attack can be automatized to retrieve any
arbitrary string from the L1-D cache.
IV. DEM O DE TAILS
The test has been carried out on an Intel R
Xeon X3470
CPU running Ubuntu Kernel 4.11.2. The two virtual machines
run on KVM version 1.3.1 and are pinned on the two
hyperthreads of a single core of the CPU, thus ensuring that
they share the L1-D cache. The VMs are plain Ubuntu kernel
3.13.0, ensuring that all L1TF mitigations shipped with the
kernel are disabled.
A part from the explanation of how the technical steps
work, the actual demo will be very simple. We will launch
a process on the victim VM that will repeatedly access a
secret byte, which in turns will therefore be stored inside the
cache. Later, we will install the offensive kernel module inside
the attacker VM. We will then perform the attack, whose
successful execution will be shown by exhibiting the secret
byte read through the L1 cache side-channel. We will also
show how continuous flushing of the L1-D cache may prevent
the attack. Obviously, this defense is questionable, as it comes
along with a significant performance penalty - many more
cache misses.
V. CONCLUSIONS & FUTURE WORK
In this work, we have demonstrated the feasibility of the
Foreshadow-VMM variant of the L1 terminal fault vulnera-
bility. As discussed in the paper, not only the attack appears
practical, but it is also very simple to perform - the attacker
only needs to deploy in the cloud an offending virtual machine
sharing a same core with a target victim!
Our current work consists in understanding the impact of
such class of vulnerabilities in the emerging 5G scenario, and
especially in Multi-Access Edge Computing contexts which
- due to their multi-access/multi-ownership nature - appear
particularly vulnerable to these new families of attacks.
Finally, while the attack has focused on virtual machines
running on an hypervisor, Network Function Virtualization
(NFV) has recently moved towards more performing ap-
proaches, such as containers. We expect that this technology
is not exempt from such types of attacks.
VI. ACKNOWLEDGMENTS
This work is partially supported by the EU Commission
in the frame of the Horizon 2020 project SPARTA (grant
#830892).
REFERENCES
[1] C. Canella, J. Van Bulck, M. Schwarz, M. Lipp, B. von Berg, P. Ortner,
F. Piessens, D. Evtyushkin, and D. Gruss, “A systematic evaluation of
transient execution attacks and defenses,” eprint arXiv:1811.05441, 2018,
November 13.
[2] P. Kocher, J. Horn, A. Fogh, , D. Genkin, D. Gruss, W. Haas, M. Ham-
burg, M. Lipp, S. Mangard, T. Prescher, M. Schwarz, and Y. Yarom,
“Spectre Attacks: Exploiting Speculative Execution,” in 40th IEEE
Symposium on Security and Privacy (S&P’19), 2019.
[3] M. Lipp, M. Schwarz, D. Gruss, T. Prescher, W. Haas, A. Fogh,
J. Horn, S. Mangard, P. Kocher, D. Genkin, Y. Yarom, and M. Hamburg,
“Meltdown: Reading Kernel Memory from User Space,” in 27th USENIX
Security Symposium (USENIX Security 18), 2018.
[4] J. Van Bulck, M. Minkin, O. Weisse, D. Genkin, B. Kasikci, F. Piessens,
M. Silberstein, T. F. Wenisch, Y. Yarom, and R. Strackx, “Foreshadow:
Extracting the Keys to the Intel SGX Kingdom with Transient Out-
of-Order Execution,” in Proceedings of the 27th USENIX Security
Symposium. USENIX Association, August 2018.
[5] O. Weisse, J. Van Bulck, M. Minkin, D. Genkin, B. Kasikci, F. Piessens,
M. Silberstein, R. Strackx, T. F. Wenisch, and Y. Yarom, “Foreshadow-
NG: Breaking the Virtual Memory Abstraction with Transient Out-of-
Order Execution,” Technical report, 2018.
[6] Y. Yarom and K. Falkner, “FLUSH+RELOAD: A High Resolution,
Low Noise, L3 Cache Side-Channel Attack,” in 23rd USENIX
Security Symposium (USENIX Security 14). San Diego, CA:
USENIX Association, 2014, pp. 719–732. [Online]. Avail-
able: https://www.usenix.org/conference/usenixsecurity14/technical-
sessions/presentation/yarom
[7] I. Corporation, “Intel 64 and IA-32 Architectures Software Developer’s
Manual,” Intel Corporation, August 2007.
[8] “Linux Page Table Managment.”
