Data Caches in Multitasking Hard Real-Time Systems

Abstract

Data caches are essential in modern processors, bridging the widening gap between main memory and processor speeds. However, they yield very complex performance models, which makes it hard to bound execution times tightly.

Full text

Page 1

Data Caches in Multitasking Hard Real-Time Systems
Xavier Vera∗†, Bj¨ orn Lisper
Institutionen f¨ or Datavetenskap
M¨ alardalens H¨ ogskola
V¨ aster˚ as, 721 23, Sweden
{xavier.vera,bjorn.lisper}@mdh.se
Jingling Xue†
School of Computer Science and Engineering
University of New South Wales
Sydney, NSW 2052, Australia
jxue@cse.unsw.edu.au
Abstract
Data caches are essential in modern processors, bridg-
ing the widening gap between main memory and proces-
sor speeds. However, they yield very complex performance
models, which makes it hard to bound execution times
tightly.
This paper contributes a new technique to obtain pre-
dictability in preemptive multitasking systems in the pres-
ence of data caches. We explore the use of cache partition-
ing,dynamiccachelockingandstatic cacheanalysisto pro-
vide worst-case performance estimates in a safe and tight
way. Cache partitioning divides the cache among tasks to
eliminate inter-task cache interferences. We combine static
cache analysis and cache locking mechanisms to ensure
that all intra-task conflicts, and consequently, memory ac-
cess times, are exactly predictable. To minimize the perfor-
mance degradation due to cache partitioning and locking,
two strategies are employed. First, the cache is loaded with
data likely to be accessed so that their cache utilization is
maximized. Second, compiler optimizations such as tiling
and padding are applied in order to reduce cache replace-
ment misses.
Experimental results show that this scheme is fully pre-
dictable, without compromising the performance of the
transformed programs. Our method outperforms static
cache locking for all analyzed task sets under vari-
ous cache architectures, with a CPU utilization reduction
ranging between 3.8 and 20.0 times for a high perfor-
mance system.
1. Introduction
The speed of processors increases faster than the
speed of memories. Cache memories are used to bridge
∗
†
Supported by VR grant no. 2001–2575.
Supported in part by Australian Research Council Grant A10007149.
this widening gap, and have become the dominant con-
straint to achieve high processor utilization.
Schedulability analysis of hard real-time systems relies
on the assumption that tasks’ worst-case execution times
(WCETs) are known. Current WCET platforms are applied
to rather simple architectures (they usually do not consider
caches) and make simplifying assumptions such that the
tasks are not preempted. In order to consider the costs of
task preemption, some studies incorporate into the schedu-
lability analysis the costs of manipulating queues, process-
ing interrupts and performing task switching [5, 6, 19, 21].
In order to get an accurate WCET, a tight worst-case
memory performance (WCMP) is needed. However, caches
introduce behaviors that are hard to predict. This leads to
an unpredictable WCMP, and thus an unpredictable WCET
as well, which jeopardizes the safety of the controlled sys-
tem. For that reason, many safety-critical systems either
do not use caches or disable them. Nevertheless, a sys-
tem with disabled caches will waste a lot of resources; the
CPU will be underutilized, and also the power consump-
tion will be larger since memory accesses that fall into the
cache consume less power than accesses to main memory.
Thus, bounding memory performance tightly in hard real-
time systems with caches is important to use the system re-
sources well.
The computation of WCET in the presence of instruc-
tion caches has progressed in such a way that it is now pos-
sible to obtain an accurate estimate of the WCET for non-
preemptivesystems [1, 2, 17]. These results can be general-
ized to preemptivesystems [3, 7, 8, 9, 23, 25, 34, 35]. How-
ever, there has not been much progress with the presence of
data caches. Instructions such as loads and stores may ac-
cess multiple memory locations (such as those that imple-
ment array or pointeraccesses), which makes the attempt to
classify memory accesses as hits/misses very hard.
We summarizebelowthe approachesthat canbeused for
analyzing WCET in the presence of data caches for multi-
tasking hard real-time systems.
1. Static Cache Analyses. They attempt to classify stat-

Page 2

ically the different memory accesses as hits or misses.
However,thebest static cacheanalysesdonotconsider
preemptive systems and are limited to codes free of
data-dependentconstructs. In addition, only results for
direct-mapped caches have been reported [22, 27, 29,
45].
2. Cache-Preemption Delays. When a task resumes its
execution,itmayspendalongtimereloadingthecache
with previously loaded cache blocks. Some studies
have addressed the issue of incorporating cache pre-
emption costs into the schedulability analysis [3, 7,
25]. However, preemption changes the cache contents
in an unpredictable manner. Thus, a cache-sensitive
analysis of a task assumed to run in isolation might
be invalid in a context where the task is preempted:
the worst-case execution path may not be the same
anymore since hits may be turned into misses and
vice versa. Adding a penalty by assuming the cache
is cold-started might be unsafe on processors with out-
of-order instruction scheduling, where a cache hit un-
der some circumstances may be more expensive than
a miss [31]. Moreover, this method resorts to a static
cache analysis to obtain the WCET.
3. Cache Locking. The ability to lock cache contents is
available on several commercial processors (PowerPC
604e [33], 405 and 440 families [18], Intel-960, some
Intel x86, Motorola MPC7400 and others). Each pro-
cessor implements cache locking in several ways, al-
lowing in all cases static locking (the cache is loaded
and locked at system start) and dynamic locking (the
state of the cache is allowed to change during the sys-
tem execution). Provided that the cache contents are
known, the time required for a memory access is pre-
dictable. Cache locking can be applied to each task in
isolation or at system startup [35].
4. Cache Partitioning. These techniques [8, 23, 28, 34]
give reserved portions of the cache to certain tasks to
guarantee that data will be in cache despite preemp-
tions, thus eliminating inter-task conflicts. The reduc-
tionof the cachesize that each task uses may,however,
translate to a loss of performance.
1.1. An Overview
If a memory access is not classified definitely as a hit
or miss, a subsequent pipeline analysis in a WCET analysis
would have to consider both situations to detect the worst-
case path. Thus, we want to guarantee an exact prediction
of hits or misses for all memory accesses.
This paper combines cache partitioning, dynamic cache
locking and static cache analysis for preemptive multitask-
ing real-time systems with data caches. Cache partitioning
allows us to eliminate inter-task conflicts, thus we can ana-
lyze each task in isolation. There are typically parts of the
code which are statically analyzable and where each ac-
cess can correctly be categorized as a cache hit or miss. For
the other parts, with data-dependent accesses (such as in-
direction arrays) or where multiple paths can be taken, we
lock the cache by inserting lock/unlock instructions in the
code.Partitioningthecachereducesthecachesize eachtask
now uses. Hence, we apply compiler optimizations such as
tiling [10, 24, 46] and padding [36, 37] to reduce the num-
ber of misses. Moreover, before locking the cache we also
load it with data likely to be accessed, thus minimizing the
possible loss of performancedue to locking.Finally, we run
the static analysis to estimate the WCMP of each task assum-
ing it has a portion of the cache, and the schedulability test
is performed.
This work makes several significant contributions. Our
previous work describes how applying cache locking dy-
namically can enhance predictability [42], but it was only
applied to non-preemptive systems. The new framework
presented in this paper applies to preemptive multitasking
systems, thereby bounding WCMP for data caches in such
systems. We have also introduced a set of transformations
tooptimizetheplacementoflock/unlockinstructions.Inad-
dition, we have written a new algorithm that applies com-
piler optimizations in concert with cache partitioning and
cache locking. Our approach yields safe and exact WCMPs;
it avoids overestimations from assumed cold-starts of the
data cache and eliminates possible safety issues, while hav-
ingalowCPU utilization.We haveimplementedoursystem
in the SUIF2 [32] compiler. Each task is compiled indepen-
dently, and only knowledge of the allocated cache partition
is required. We do not rely on any specific hardware com-
ponent,thus its applicability is not limited to a particularar-
chitecture.
The rest of the paper is organized as follows. Section 2
introduces the program model, cache architecture and task
model used in our approach. We review the static analy-
sis in Section 3. Section 4 discusses the transformations to
havea staticallypredictabledata cacheformultitaskingsys-
tems. Section 5 presents our experimental framework, and
Section 6 discusses our results. Section 7 reviews some re-
lated works. Finally, we conclude and give a road map to
future extensions in Section 8.
2. Terminology
2.1. Program Model
We consider programs consisting of subroutines, calls,
arbitrarily nested but well-structured loops, and assign-
ments possibly guided by IF conditionals. Our method can

Page 3

be extendedto unstructuredcode, but for simplicity we stay
with this program model.
In this paper, all programs are written in C. Thus, all ar-
rays are assumed to be in row major order. The following
restrictions define the scope of programs where we can ap-
ply our static analyzer[42] without any transformation,that
is, those programs where only one path is analyzed:
• Calls are non-recursive.
• Bounds of all loops are known and affine.
• The IF conditionals are analyzable at compile time.
In order to ensure that the static analysis [15, 41, 42] can
be done in the polyhedral model [12], we add the following
constraint:
• The subscript expressions of array references are
affine.
We rely on the compiler to identify compile-time and
run-time constants. We use standard compile-time tech-
niques such as constant propagation to detect more con-
stants, which allows more expressions to be analyzed stati-
cally. To address the symbolic loop bound problem, we use
interprocedural constant propagation to eliminate as many
symbolic loop bounds as possible. This may also be help-
ful to know statically which recursive calls are made,
thus enlarging the scope where the static analysis is ap-
plied.
Otherwise, when multiple paths are possible, we apply
path merging in order to reduce the number of paths be-
ing analyzed. We assume that the maximum number of it-
erations of a loop and the maximum number of recursive
calls are known. This can be done by either manual annota-
tions [11] or automatic approaches [16].
2.2. Cache Model
We consider a uniprocessor with a two-level mem-
ory hierarchy consisting of a virtually-indexed K-way
set-associative data cache using LRU replacement pol-
icy followed by main memory. Note that this cache archi-
tecture is assumed in all the existing analytical methods
reviewed in Section 7.
InaK-wayset-associativecache,eachcacheset contains
K cache lines. Let Cs(Ls) be the cache (line) size in bytes.
The total numberof cache sets is thus Cs/(Ls×K). A cache
is called direct-mapped when K=1, and fully-associative
when K=Cs/Ls.
Inordertouse cachelocking,we assume thatthereexists
a locking mechanism that allows a cache line to be locked.
Such mechanism is available in several modern processors
such as PowerPC 604e [33], 405 and 440 families [18],
Intel-960,some Intelx86andMotorolaMPC7400.Besides,
we assume that the processors offer the ability to load and
invalidate cache lines selectively. Otherwise, both of them
can be “simulated” on software at a cost of some perfor-
manceloss. Ourapproachassumesthat cachepartitioningis
implemented either by a hardware or software means. The
partition unit is a cache set.
2.3. Task Model and Schedulability Analysis
We consider a set of N periodic tasks Ti, 1 ≤ i ≤ N.
We denote the period and worst-case execution time of task
Tiby Piand Ci, respectively.
We consider two schedulability analyses for periodic
tasks, UA (utilization-based analysis) and RTA (response
time analysis). For dynamic priority preemptively sched-
uled systems (e.g., earliest deadline first), the utilization
condition U ≤ 1 is necessary and sufficient, where U is de-
fined as follows:
U
=
N
?
i=1
Ci
Pi
(1)
For static priority preemptively scheduled systems such
as rate monotonic, we use response time analyses [20, 38]
to obtain a necessary and sufficient condition. For a task Ti,
the idea is to consider all preemptions produced by higher
prioritytasks onanincreasingwindowtime.Thefixedpoint
of the following recurrence gives the response time Riof
task Ti:
R0
i
=
...
Ci
(2)
Rn+1
i
=
Ci+
?
Tj∈HP(Ti)
?Rn
Pj
i
?
× Cj
where HP(Ti) is the set of tasks with higher priority than
Ti. In order to check the schedulability of task Ti, one only
hastocomparetheresponsetime Riwith its periodPi. Task
Tiis schedulable if and only if Ri≤ Pi.
Our approach eliminates cache penalties due to cold-
starting the cache after a context switch. Thus, classical
non-cache sensitive schedulability analyses should be used
rather than their cache-sensitive versions, CUA [3] and
CRTA [7].
3. CMEs Overview
CMEs [15] are mathematical formulas that provide a
precise characterization of the cache behavior for perfectly
nested loops consisting of straight-line assignments. In or-
der to describe data reuse, we use an extension [44, 48] of
the well-known concept of reuse vectors [46], which de-
scribes the most recent previous access (MRPA) among ar-
bitrary loop nests.

Page 4

Based on the description of reuse given by reuse vec-
tors, some equations are set up that describe those iteration
points where the reuse is not realized. Solving them gives
informationabout the numberof misses and where they oc-
cur.Inapreviouswork[44],wefurtherextendedtheminor-
der to make whole program analysis feasible, by handling
call statements, IF statements and arbitrarily nested loops.
Write Policy. Given a memory reference, the equations
are to investigate whether the reuse described by its reuse
vectors is realized or not. We now briefly discuss how we
extend our analysis to caches with write-no-allocate fea-
tures.
The idea consists of treating in a different manner the
reuse vectors correspondingto the reuses of data previously
accessed by write references [14]. For a direct-mapped
cache, the solution is as simple as ignoring all these reuses.
For set-associative caches, we modify the solver so a write
access that misses does not modify the LRU algorithm.
D-TLB. TLB is a hardwaretable offrequentlyused page
translations. It is usually implemented as a set-associative
cache, which is indexed with a subset of the bits that form
the virtual address. Thus, in order to simulate its behavior
we only have to compute the MRPAs [48] for each memory
access in terms of pages instead of memory lines.
Multilevel Caches. For these architectures, we have to
analyze differently memory references depending on the
cache level they are accessing [40]. For that purpose, a set
of equations is set up for each of the cache levels. When
analyzing potential cache set contentions, only memory ac-
cesses that miss in lower cache levels are considered. Thus,
we can see the equationsforeach level as filters, where only
thosememoryaccesses that miss are analyzedin higherlev-
els.
While the results are accurate, safety is not guaranteed
for systems with unified caches or where the upper levels
of cache are physically indexed. An extension where the ef-
fects of instructions on unified caches and the use of physi-
cal addresses are considered is left as future work.
4. Obtaining a Predictable System
When considering cache memories, schedulability anal-
yses should consider the cost of reloading the cache lines
that may have been evicted from cache. When a preempted
taskresumesits execution,it mayspendalot oftimereload-
ing those cache lines that have been displaced from cache.
Recent studies incorporate some cache-related preemption
costs into the schedulability analysis [3, 7, 25]. They basi-
cally consider that the preempted task will incur a miss for
each cache line when resuming execution. However, this
approach cannot be used when dynamic cache locking is
used, since the cost of preempting a task that is accessing a
locked region may be much larger than a cache miss for ev-
ery cacheline. A preemptingtask may unlockthe cache and
load it with its own data; when the preempted task resumes
its execution, it will not reload the cache since the cache is
locked. Thus, there may be more extra misses than one per
cache line throughout the locked region.
Our goal is to have a methodthat allows obtainingan ex-
act (we want to guarantee an exact classification of mem-
ory accesses as cache hits or misses) and safe WCMPs of
tasks formultitasking systems with data caches, so that cur-
rent schedulability analyses can be applied without modifi-
cations.
PredictMultiTask given in Figure 1 takes as input a set of
tasks and a cache architecture, and generates a set of cache
partitions and a set of analyzable tasks that have the same
semantics as the original tasks. Then, we run our static an-
alyzer [42] which calculates an exact WCMP for each trans-
formed task. In this section, we explain in detail the dif-
ferent parts of the algorithm. We first discuss the implica-
tions of using the cache partitioning technique. Then, we
review our solution to the problem of predictability for data
caches. Finally, we discuss the application of different tech-
niques to optimize the cache behavior of tasks, so that the
performance is not jeopardized.
4.1. Cache Partitioning
Inter-task interferenceoccurs when cache lines from dif-
ferent tasks conflict in cache, which causes unpredictabil-
ity. Cache partitioning [23] divides the cache into disjoint
partitions, which are assigned to tasks in such a way that
inter-conflicts are removed.
Let {T1,...,Tn} be a set of tasks. Usually, cache par-
titioning creates n + 1 partitions, one for each real-time
task and another one which is shared among non-real-time
tasks. Each task is only allowed to access its own partition,
thus removing inter-task conflicts. Note that tasks that have
the same priority (thus, they are non-preemptively related
to each other) can share the same partition, since they are
only preempted by tasks that have higher priority, and thus
the predictability of cache behavior is not affected. There-
fore, it is enough to divide the cache in p partitions, where
p is the number of different priorities.
Cache partitioning can be implemented either in soft-
ware [34, 47] or hardware [23]. Both techniques impose
the partition size to be a power of two, so that the pointer
transformation to access data structures can be performed
in a fast way.1The software approach requires compiler
andlinkersupport[34], whichareresponsibleforrelocating
data to provide exclusive mappings on the cache for each
task.
1This restriction does not apply to instruction caches.

Page 5

INPUT
S
C
=
=
a set of tasks
a cache architecture
OUTPUT
PredictMultiTask(S, C)=
<set of tasks, set of partitions>
ALGORITHM
CP:= CreatePartitions(S, C);
S aux := ∅;
for each task Ti ∈ S
CPiis Ti’s cache partition
P aux:= LockUnpredictableRegions(Ti);
P aux:= OptimizeLock(P aux);
P aux:= LoadData(P aux);
P aux:= CacheOptimize(P aux, CPi);
S aux.insert(P aux);
// set of partitions
// set of modified tasks
// modified task after locking
PredictMultiTask(S, C) := < S aux, CP >
Figure 1. An algorithm for obtaining a predictable set of tasks on a multitasking system.
When a cache is partitioned, each task will access a
smaller fraction of the cache, which may cause capacity
misses toincrease.Thus,thesize ofthepartitionshas anim-
pact on the overall performance. In order to obtain the best
data cache partitioning, the decision should be taken based
ontheprioritiesandthereusepatternsoftasks.Forinstance,
a task that has a workload of 8KB but only accesses each
cache line once only needs one cache line, whereas a task
witha workloadof 1KB that reuseseach cacheline onemil-
lion times would suffer a performance loss with a partition
smaller than 1KB.
Our approach works with both hardware and software
mechanisms, and it does not depend on the size of the par-
titions created. From now on we assume that the cache
is divided in n equally-sized partitions, one for each task.
Yet simple, we show how it is good enough for schedul-
ing real sets of tasks and better utilizing the CPU than
otherapproaches.An algorithmtoobtainevenbetterperfor-
mance,bychoosingdifferentcachepartitionsizesfordiffer-
ent tasks, is left as future work.
4.2. Obtaining a Predictable Program
A practical limitation for WCET estimation is that the
numberofpathstobeanalyzedcaneasilybeprohibitive,es-
pecially when studying loop constructs with multiple paths
inside. Thus, we use path merging to reduce the path explo-
sion by mergingpaths in those cases where a path enumera-
tion is needed [13, 17, 30]. This includes data-dependent
conditionals, loops with multiple paths inside and loops
with unknown loop bounds.
On the other hand, there are data-dependentmemory ac-
cesses. This includes indirection arrays (e.g., a[b[i]], where
b[i] is not statically known), variables allocated dynami-
cally (e.g., mallocs) and pointer accesses that cannot be de-
termined statically. We also include nonlinear array refer-
encesthatarenothandledbyourstatic analyzer(e.g.,a[i*j])
and library and operating system calls.
Both situations lead to unpredictability. Merging paths
causes an unknown state of the cache, since a new state of
thecacheis createdbasedonthestate ofthecacheatthe end
of each path [1, 30]. Data-dependent memory accesses can-
not be classified definitely as hits/misses and also cause an
unknown state of the cache.
We avoidthis sourceof unpredictabilityby lockingthose
parts of the code where paths are merged or have data-
dependent memory accesses [42]. LockUnpredictableRe-
gions makes use of the control-flow graph (CFG) of the
program, and inserts lock/unlock instructions (lock/unlock
nodes in the control-flow graph) when necessary. In our
approach, we always apply lock/unlock instructions to the
whole cache. We use the code in Figure 2(a) to illus-
trate how LockUnpredictableRegionsworks. It traverses the
control-flow graph of the program and detects the two con-
structs, in a well-structured CFG, that might give rise to
multiplepathsandthusneedspathmerging(data-dependent
conditionals, and loop nests with unknown numberof itera-
tions). Besides, it detects data-dependent memory accesses
which are non-analyzable. Figure 2(b) shows the code af-
ter the lock/unlock instructions have been inserted. Region
1 is created due to the non-analyzable memory access. Re-
gions 2 and 2.1 are created because of the path merging
needed there.
4.2.1. Optimizing Placement of Lock/Unlock Instruc-
tions Placementoflock/unlockinstructionsmayaffectper-
formance; (i) the execution of lock/unlock instructions in-