A table-based method for single-pass cache optimization.

Full text

Page 1

A Table-based Method for Single-Pass Cache Optimization
Pablo Viana
Federal University of Alagoas
Arapiraca-AL, Brazil
pablo@lccv.ufal.br
Edna Barros
Federal University of Pernambuco
Recife-PE, Brazil
ensb@cin.ufpe.br
Ann Gordon-Ross
University of Florida
Gainesville-FL, USA
ann@ece.ufl.edu
Frank Vahid
University of California, Riverside
Riverside-CA, USA
vahid@cs.ucr.edu
ABSTRACT
Due to the large contribution of the memory subsystem to total
system power, the memory subsystem is highly amenable to cus-
tomizationforreduced power/energy and/or improvedperformance.
Cache parameters such as total size, line size, and associativity can
be specialized to the needs of an application for system optimiza-
tion. In order to determine the best values for cache parameters,
most methodologies utilize repetitious application execution to in-
dividually analyze each configuration explored. In this paper we
propose a simplified yet efficient technique to accurately estimate
the miss rate of many different cache configurations in just one
single-pass of execution. The approach utilizes simple data struc-
tures in the form of a multi-layered table and elementary bitwise
operations to capture the locality characteristics of an application’s
addressing behavior. The proposed technique intends to ease miss
rate estimation and reduce cache exploration time.
Categories and Subject Descriptors
B.3[MemoryStructures]: PerformanceAnalysisandDesignAids
General Terms
Algorithms
Keywords
Configurable cache tuning, cache optimization, low energy.
1.INTRODUCTION
Optimizationofsystemperformance andpower/energyconsump-
tion is an important step during system design and is accomplished
through specialization, or tuning, of the system. Tunable parame-
ters include supply voltage, clock speed, bus width and encoding
schemes, etc. Of the many tunable parameters, it is well known
that one of the main bottlenecks for system efficiency resides in
the memory sub-system (all levels of cache, main memory, buses,
etc) [16]. The memory subsystem can attribute to as much as 50%
of total system power [1, 17].
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
GLSVLSI’08, May 4–6, 2008, Orlando, Florida, USA.
Copyright 2008 ACM 978-1-59593-999-9/08/05 ...$5.00.
Memory subsystem parameters such as total size, line size, and
associativity can be tuned to an application’s temporal and spatial
locality to determine the best cache configuration to meet optimiza-
tion goals [3]. However, the effectiveness of such tuning is de-
pendent on the ability to determine the best cache configuration to
complement an application’s memory addressing behavior.
To determine a cache size that yields good performance and low
energy for an application, the size must closely reflect the temporal
locality needs of an application. It is important to determine how
frequently memory addresses are accessed and how long it takes
for an executing application to access the same memory reference
again. This property is mostly attributed to working-set character-
istics such as loop size.
Similarly, the cache line size must closely reflect the spatial lo-
cality of an application, which is present in straight-line instruc-
tion code and data array accesses. Additionally, associativity must
closely reflect the needs of the application.
Todetermine thebest values for these tunable parameters, or best
cache configuration, existing cache evaluation techniques include
analytical modeling [6, 10] and execution-based evaluation [4] to
evaluate the design space. Analytical models evaluate code char-
acteristics and designer annotations to predict an appropriate cache
configuration in a very short amount of time, requiring little de-
signer effort. Whereas this method can be accurate, it can be diffi-
cult to predict how an application will respond to real-world input
stimuli.
A more precise technique is execution-based evaluation. In this
technique, an application is typically simulated multiple times, and
through theuseofacachesimulator, applicationperformance and/or
energyareevaluatedfor eachcacheconfiguration explored. Whereas
this technique is more accurate than an analytical model, modern
embedded systems are becoming more and more complex and sim-
ulating these applications for numerous cache configurations can
demand a large amount of design time. To accelerate execution-
based evaluation, specialized caches have been designed that allow
for cache parameters to be varied during runtime [2, 14, 19]. How-
ever, due to the intrusive nature of the exploration heuristics, the
cache must be physically changed to explore each configuration.
Exploring a large number of cache configurations can poten-
tially significantly adversely effect program execution in terms of
energy and performance overhead while exploring poor configura-
tions. To reduce the number of configurations explored, efficient
heuristics have been proposed [8, 19] to systematically traverse the
configuration space and result in a near-optimal cache configura-
tion while evaluating only a fraction of the design space. However,
even though the number of cache configurations is greatly reduced,
in some systems, tens of cache configurations may need to be ex-
plored, thus still potentially imposing a large overhead and con-

Page 2

suming too much exploration time. Exploration time must be quick
enough to adapt to rapidly changing resource requirements [7].
Instead of changing the cache configuration numerous times to
evaluate eachdifferentcache configuration, muchinformationabout
the memory addressing behavior could be extracted from a single
execution of an application independent of the cache configuration.
In multi-cache evaluation, multiple cache configurations are eval-
uated in a single pass of execution [18]. For example, the number
of neighboring addresses that are accessed in a short period of time
or how often a given address is repeatedly referenced can suggest
an application’s temporal and spatial locality requirements. If such
properties about the spatial and temporal locality of the applica-
tion are extracted and well correlated in an organized structure, the
behavior of many cache configurations can be estimated and appro-
priate cache parameters can be projected.
In this work, we present a simplified, yet efficient way to extract
locality properties for an entire cache configuration design space in
just one single-pass. SPCE (pronounced spee-cee), our single-pass
multi-cache evaluation methodology, utilizes small, compact table
structures and elementary bitwise operations consisting of compar-
isons and shifting to allow us to estimate the cache miss rate for all
configurations simultaneously. SPCEprovides design time acceler-
ation in a simulation-based environment, but most importantly, we
design SPCE with a hardware implementation in mind, providing
an important non-intrusive cache exploration alternative for quick
runtime exploration. In this paper, we present a detailed algorithm
for SPCE’s operation and evaluate SPCE in a simulation-based en-
vironment compared to a state-of-the-art cache tuning heuristic.
In [9], we provide a hardware implementation of SPCE and quan-
tify the importance of such a runtime tuning environment.
This paper is organized as follows: Section 2 discusses related
work pertaining to single-pass cache evaluation.
scribes an overview of SPCE. In the Section 4, we present ele-
mentary properties for addressing behavior analysis to estimate the
cache miss rate of fully-associative caches. Section 5 extends those
properties to analyze address conflicts and introduces more ad-
vanced concepts to build a multi-layered table for multi-cache eval-
uation of direct-map caches as well as set-associative caches. Sec-
tion 6 presents the SPCE algorithm and discusses its implemen-
tation details. In Section 7 we validate SPCE with experimental
results and finally in Section 8, we conclude the paper and outline
future directions for SPCE to a broader domain.
Section 3 de-
2.RELATED WORK
Much research exists in the area of multi-cache evaluation, how-
ever, nearly all existing techniques require multiple passes to ex-
plore all configurable parameters or employ large and complex data
structures that are not amenable to hardware implementation, thus
restricting their applicability to strictly a simulation-based evalua-
tion environment.
Algorithms for single-pass cache simulation tackle the problem
of multi-cacheevaluationbyexamining concurrently asetof caches
with different sizes during the same execution pass. Research on
this issue began in 1970 when Mattson et al. presented an algo-
rithm for simulating fully-associative caches with varying sizes and
a fixed block size [15]. The algorithm utilized stack-based simula-
tion and took advantage of the inclusion property.
The inclusion property states that at any time, the contents of
a cache are a subset of the contents of a larger cache. Hill and
Smith [12] identified the set-refinement property, which extends
the inclusion property to study the inclusion effects of cache as-
sociativity, and extended the inclusion property for direct-mapped
and set-associative caches. Sugumar and Abraham [18] developed
algorithms using binomial trees resulting in single-pass schemes 5
times faster than previous approaches. They also proposed another
single-pass algorithm to simulate caches with varying block sizes.
In [5], Cascaval and Padua proposed a method to estimate cache
misses at compile time using a machine independent model based
on a stack algorithm.
Most of the existing methods perform very well for a given set
of caches with a fixed line size or a fixed total size. Thus, these
algorithms can be utilized in simulation-based cache tuning, reduc-
ing the number of necessary simulation passes. However, since
these methods are unable to evaluate all different cache parameters
simultaneously, multiple simulation passes are still required, thus
in a runtime tuning environment, cache exploration could be too
lengthy. In [13], Janapsatya et al. present a technique to evaluate
all different cache parameters simultaneously and, tothe best of our
knowledge, is the only such technique. They present a tree-based
structure consisting of multiple linked lists to keep track of cache
statistics for a large design space. Whereas this work most closely
resembles our methodology and shows tremendous speedups in
simulation time, their methodology was not designed with a hard-
ware implementation in mind. Our methodology utilizes simple ar-
ray structures, structures that are more amenable to a light-weight
hardware implementation.
3. SPCE OVERVIEW
SPCE is a single-pass multi-cache evaluation technique to eval-
uate all values for all cache parameters (total size, line size and
associativity) simultaneously, requiring only one simulation pass.
Whereas previous single-pass cache evaluation techniques utilize
complex data structures to estimate cache miss values, SPCE uti-
lizessimple tablestructuresand elementary bitwiseoperations con-
sisting of comparisons and shifting.
Figure 1 illustrates an overview of SPCE’s cache evaluation me-
thodology. The running application ai produces a sequence of in-
struction addresses T, which can be captured independently of the
cache configuration, by using an instruction-set architecture simu-
lator or executable platform model [4]. In this work, we utilize a
processor simulator to generate an address trace for post execution
processing, but we point out that, given the sheer size of typical ad-
dress traces, address trace generation may be omitted and instruc-
tion addresses may be trapped during simulation and fed to SPCE
in parallel.
Cache configuration
design space C={c1,…cm}
Locality and Conflict
Multi-layered Table Generation
Running
Application (ai)
Sequence of
addresses T
Cache Miss Rate
on c1
Cache Miss Rate
on c2
...
Cache Miss Rate
on cm
Multi-cache
evaluation
Stack-based
analysis
T[ t ]
...
T[3]
T[2]
T[1]
T[0]
SPCE
L
}K
Figure 1: SPCE overview: multi-layered table generation for
multi-cache evaluation.
SPCE processes the sequence of addresses and analyzes the en-
tire cache configuration design space C consisting of m different

Page 3

configurations. Each configuration is a unique combination of val-
ues for line size, associativity, and total size in the design space.
SPCE uses a stack structure to store previous addresses for cache
hit evaluation. When an address is processed, SPCEscans the stack
to determine if the current address has been fetched previously and
would perhaps result ina cache hit depending on the cache configu-
ration. After SPCE processes an address, if the address was already
present in the stack, the address is removed and pushed onto the top
of the stack. If the address was not present in the stack, it is simply
pushed onto the top of the stack.
We present details of data generation in sections 4 and 5. The
resulting locality information for line size and total cache size anal-
ysis populates the table L, whileconflict data for associativity anal-
ysis populates the multi-layered table K, where each layer repre-
sents the associativity levels explored.
After processing all instruction addresses, the cells of the tables
L and K store the number of cache hits for the sequence of ad-
dresses analyzed for all cache configurations in the space C. By
summing up the contents of the tables and subtracting that value
from the size of the address trace, it is easy to evaluate cache miss
rates for every cache configuration. The miss rates can then be sup-
plied to an energy model for the system to calculate energy con-
sumption for each cache configuration.
4.LOCALITY ANALYSIS
4.1Definitions
Whileapplication aiexecutes, thememory hierarchy fulfillssuc-
cessive instruction reference requests. We define the time ordered
sequence of referenced addresses by the vector?T[t], t ∈ Z+(t is
a positive integer), of length |?T|, such that?T[t] is the tthaddress
referenced [11].
In a traditional cache mapping, two addresses?T[ti] and?T[ti+d]
belong to the same cache block if, and only if
where?T[ti + d] is the dthaddress referenced after?T[ti] and 2b
is the cache block size in number of words (words are defined in
Bytes).
Since the block size is the number of words, usually a power of
2, it is reasonable to represent it by the power 2b. We define the
operator ⊲ as the bitwise shift operation?T[ti]⊲b where the address
?T[ti] is shifted to the right b times. Shifting?T[ti] to the right b
times is equivalent to dividing?T[ti] by 2b(Equation 1).
? T[ti]
2b
=
?T[ti+d]
2b
,
?T[ti] ⊲ b =
?T[ti]
2b
(1)
Thus, if?T[ti] ⊲ b =?T[ti + d] ⊲ b, then the addresses?T[ti]
and?T[ti+ d], are references to the same cache block of 2bwords.
Note that for the particular case where b = 0,?T[ti] and?T[ti +
d] would correspond to exactly the same address. By considering
the probability of frequent accesses to the same block during the
execution of a given application, spatial and temporal locality can
be directly correlated as a function of b and d.
We evaluate the locality in the sequence of addresses?T[ti] of a
running application ai by counting the occurrences where?T[ti] ⊲
b =?T[ti+d]⊲b and registering it in the cell L(b,d) of the locality
table. The number of rows and columns of the locality table are
defined by the boundaries 1 ≤ d ≤ dmax and 0 ≤ b ≤ bmax
respectively (Table 1) where dmax is defined as the total number
of available lines in the largest cache of the configuration space,
and bmax defines the total number of line sizes available.
The locality table L(b,d) represents a structured abstraction of
a sequence of addresses?T and can be built by evaluating the vari-
ables b and d when?T[ti]⊲b =?T[ti+d]⊲b, as the application runs.
It is important to note that when evaluating?T[ti]⊲b =?T[ti+d]⊲b,
it isnot sufficient to simply count the number of references d which
occur between?T[ti] and?T[ti+ d].
If any reference?T[ti + j] for 1 ≤ j < d results in a cache
hit, that reference would not displace any cache line. Thus, in this
situation,?T[ti] ⊲ b =?T[ti + d + 1] ⊲ b would result in a cache
hit as well. As this point we redefine d as the delay or the number
of unique cache references occurring between any two references
where?T[ti] ⊲ b =?T[ti+ d] ⊲ b.
Table 1: Locality table L(b,d)
L(0,1)
L(0,2)
L(0,3)
...
L(0,dmax)
L(1,1)
L(1,2)
L(1,3)
...
L(1,dmax)
L(2,1)
L(2,2)
L(2,3)
...
L(2,dmax)
...
...
...
...
...
L(bmax,1)
L(bmax,2)
L(bmax,3)
...
L(bmax,dmax)
4.2Fully-associative Cache Miss Rate
In a fully-associative cache, references can be mapped to any
cache line. For this reason, a cache with n lines, using an optimal
(OPT) or near optimal (LRU) replacement policy stores any ad-
dress reference?T[ti] until address?T[ti+n] is referenced, where n
represents the number of cache misses occurring between reference
?T[ti] and reference?T[ti+ n].
A fully-associative cache configuration isdefined by the notation
cj(b,n), where b defines the line size in terms of words, and n the
total number of lines in the cache.
We know that L(b,d) gives us the number of occurrences?T[ti]⊲
b =?T[ti + d] ⊲ b in a sequence of addresses?T. Thus, L(b,d)
indicates how often the same block is regularly accessed after d
references to other addresses.
Weconclude thatafullyassociativecacheconfiguration cj(b,n),
composed of n lines with 2bwords per line, is able to hold any ref-
erence?T[ti] as long as it is repeatedly accessed, yielding L(b,d)
cache hits, ∀d ≤ n. (Equation 2).
Cache hit[cj(b,n)] =
n
?
d=1
L(b,d)
(2)
We can estimate the cache miss rate of a given cache configura-
tioncj(b,n) forasequence?T bysubtractingtheresultofCache hit
from the total size of the sequence of addresses |?T| (Equation 3):
Miss rate[cj(b,n)] =|?T| − Cache hit[cj(b,n)]
|?T|
(3)
Any fully-associative cache configuration cj(b,n) within the de-
sign space defined by the boundaries 0 ≤ b ≤ bmax and 1 ≤ d ≤
dmax can be estimated by using the Locality Table L. Thus, just
one single-pass simulation of the application (or trace?T) is neces-
sary to generate the entire contents of L.

Page 4

5.DIRECT-MAPPED AND SET-ASSOCIA-
TIVE CACHES
5.1Multi-layered Conflict Table
The locality table L(b,d) presented in Section 4 composes an
efficient way to estimate the cache miss rate of fully-associative
caches. However, due to the high cost in terms of area and energy
consumption for fully-associative caches, most real-world cache
devices are built as direct-map or set-associative structures.
For these cache structures, the locality table L(b,d) can not be
usedtoestimatecachemisses, sincemapping conflictsfor addresses
that map to the same cache line are not considered. Miss rate esti-
mation for such cache structures must take into account the evalu-
ation of address mapping conflicts.
Here, we define s as the number of sets independent of the as-
sociativity. A direct-mapped cache can be considered a particular
case of a set-associative cache (set size is 1 line). In this case, the
number of sets is equal to the number of cache lines (s = n).
Two distinct addresses?T[ti] and?T[ti+d] suffer from an address
mapping conflict if?T[ti]⊲b divided by s and?T[ti+d]⊲ b divided
by s results in the same remainder (Equation 4).
(?T[ti] ⊲ b) mod s = (?T[ti+ d] ⊲ b) mod s =⇒ Conflict (4)
In fully-associative caches (s = 1), the probability of a given
reference (?T[ti] ⊲ b) being present in the cache is dependent on the
number of cache lines and how long it takes until the same block is
referenced again (d ≤ n).
In direct-mapped and set-associative caches, the probability of a
given reference being present in the cache depends on the number
of cache sets (s) and how many cache conflicts occur before the
same blockisreferenced again. If thelevel of associativityishigher
than the number of conflicts, the reference will still be in the cache
(Hit). For the analysis of cache conflicts, we propose the set of
table layers Kα, denoted as a conflict table, which is composed
of α layers, one for each associativity explored, as illustrated in
Table 2.
Table 2: Conflict table Kα(b,s) for α = 1 and 2
K1(0,1)
K1(0,2)
K1(0,4)
...
K1(0,smax)
K1(1,1)
K1(1,2)
K1(1,4)
...
K1(1,smax)
K1(2,1)
K1(2,2)
K1(2,4)
...
K1(2,smax)
...
...
...
...
...
K1(bmax,1)
K1(bmax,2)
K1(bmax,4)
...
K1(bmax,smax)
K2(0,1)
K2(0,2)
K2(0,4)
...
K2(0,smax)
K2(1,1)
K2(1,2)
K2(1,4)
...
K2(1,smax)
K2(2,1)
K2(2,2)
K2(2,4)
...
K2(2,smax)
...
...
...
...
...
K2(bmax,1)
K2(bmax,2)
K2(bmax,4)
...
K2(bmax,smax)
SPCE builds the conflict table Kα for a running application by
analyzing the sequence of addresses?T[t] and counting the num-
ber of occurrences (mapping conflicts) of distinct address blocks
mapped tothe same cache line. The number of lines in each layer α
of the conflict table is defined by the maximum number of sets s =
smax. For any cache configuration (b,s), Equation 4 computes the
number of mapping conflicts (x) between?T[ti]⊲b and?T[ti+d]⊲b,
which defines the appropriate layer α to fill in Kα(b,s). The layer
α isdetermined by rounding up the value of x+1to the next power
of 2 (Equation 5).
α = 2⌈log2(x+1)⌉
(5)
5.2Miss Rate Estimation
We designed the conflict table Kα(b,s) in a such way that layer
α refers to the associativity of a given cache configuration. Layer
α = 1, for example, characterizes the addressing behavior of a
direct-map cache, layer α = 2 characterizes a two-way set-asso-
ciative cache, layer α = 4 characterizes a four-way cache, and so
on.
At the end of simulation, the value stored in each element of the
tableKα(b,s)indicateshow manytimesthesameblock (size2b) is
repeatedly referenced and results in a hit. Depending on the cache
mapping, which is defined by the number of sets (s), the lowest
associativity level which guarantees Kα(b,s) cache hits is given
by α.
A given cache configuration with level of associativity w is ca-
pable of overcoming no more than w − 1 mapping conflicts. Thus,
the number of cache hits is determined by summing up the cache
hits from layer α = 1 up to its respective layer α = w, where w
refers to the associativity. From now on, each cache configuration
will be defined by cj(w,b,s) (Equation 6).
Cache hit[cj(w,b,s)] =
w
?
α=1
Kα(b,s)
(6)
6. ALGORITHM IMPLEMENTATION
Figure 3 shows the SPCE algorithm to build the multi-layered
conflict table. A stack keeps track of the sequence of previously ac-
cessed addresses and is repeatedly scanned to evaluate which con-
figurations would result in that access being a “cache hit.”
process(ADDR)
{
ADDR = ADDR >> W
for B = BMAX downto BMIN{
//shift out block offset
BASE_ADDR = ADDR >> B
//scan stack
WAS_FOUND = lookup_stack(BASE_ADDR)
if(WAS_FOUND){
for S = SMIN to SMAX{
//scan stack looking for set conflicts
NUM_CONFLICTS = count_conflicts(S, BASE_ADDR)
if(NUM_CONFLICTS <= AMAX){
//mark the appropriate table level
ALPHA = roundup(NUM_CONFLICTS)
update_table(ALPHA, S, B)
}
} // end for S = SMIN to SMAX
}
} // end for B = BMAX downto BMIN
//push or move addr to top
update_stack(WAS_FOUND, ADDR)
}
//shift out word offset
//for each line size
//for each set size
Figure 2: The SPCE algorithm.
When the address is found in the stack, SPCE analyzes the num-
ber of set conflicts to determine the minimum set-associativity to
yield ahit. The occurrences are registeredinto theappropriate layer
of the conflict table by incrementing the value of its cell. After hit
determination, if the address was found in the stack, the address is
moved to the top of the stack. Otherwise, the new address is pushed
onto the stack. Taking advantage of the inclusion property of a

Page 5

larger cache line size over a smaller size, the outer loop responsible
for the line size exploration was intentionally implemented with a
decrementing counter to optimize stack scanning.
Pre-defining the bounds of the tables specify the particular de-
signspaceof cacheconfigurations explored. Thenumber of columns
in each table layer is defined by the minimum and maximum num-
ber of words per line (2bminand 2bmax, respectively). The number
of lines in the tables are defined by the minimum and maximum
number of cache sets (smin and smax, respectively) in the cache
configuration space. The number of table layers is dependent on
the highest associativity level considered (amax).
Eachcacheconfiguration inthedesign spaceisdefined bythepa-
rameters w,b, and s (associativity, line size, and number of sets, re-
spectively), which determine their corresponding cells in the multi-
layered table. The number of misses for any cache configuration
can be easily calculated by summing up the table cells for the given
configuration todetermine thenumber of cache hits and subtracting
the amount from the total number of addresses.
7.EXPERIMENTAL RESULTS
7.1Setup
We implemented SPCE as a standalone C++ program to process
an instruction address trace file. We gathered instruction address
traces for 9 arbitrarily chosen applications from Motorola’s Power-
Stone benchmark suite [14] using a version of SimpleScalar mod-
ified to output instruction traces. SimpleScalar is a 64-bit archi-
tecture, thus each address will be shifted by 8 to remove the word
offset.
Although SPCE does not impose any restriction on the param-
eters of the configurable cache architecture, in order to determine
both theaccuracy and simulation timespeedup compared toa state-
of-the-art cachetuningheuristic, weadoptedtheconfigurable cache
architecture presented by Zhang et al [20]. Zhang’s system archi-
tecture consists of a cache hierarchy with separate level one in-
struction and data caches, both connected to the main memory and
a cache tuner connected to both caches.
Using specialized configuration registers, each cache offers con-
figurable size, line size, and associativity. To offer configurable
size and associativity, each cache is composed of four configurable
banks each of which acts as a way in the cache. The base cache is
a 4-way set-associative cache. The ways/banks may be selectively
disabled or enabled to offer configurable size. Additionally, ways
may be logically concatenated to offer direct-mapped, 2-way, and
4-way set-associativities.
Given the bank layout of the cache, some size and associativity
combinations are not feasible. The cache offers a base physical
line size of 16 bytes with configurability to 32 and 64 bytes by
fetching/probing subsequent lines in the cache. According to the
hardware layout verification presented by Zhang et al. in [20], for
their configurable cache, the configurability does not impact access
time.
The configurable cache offers the set of m = 18 distinct config-
urations shown in Table 3, where each configuration is designated
by a value cj. For example, a 4-kByte direct-mapped cache with
a 32-byte line size is designated as c8. These designations will be
used throughout the rest of this paper to identify each particular
cache configuration.
The values of w,b, and s characterize the configurations with re-
spect to SPCE and are determined as follows: w refers to the asso-
ciativity of the configuration, which for the configurable cache uti-
lized, is limited to directed-map, 2-way, and 4-way set-associative
caches, defined by the values w = 1, 2, and 4 respectively.
cj
c1
c2
c3
c4
c5
c6
c7
c8
c9
c10
c11
c12
c13
c14
c15
c16
c17
c18
description
(w,b,s)
1, 1, 128
1, 1, 256
2, 1, 128
1, 1, 512
2, 1, 256
4, 1, 128
1, 2, 64
1, 2, 128
2, 2, 64
1, 2, 256
2, 2, 128
4, 2, 64
1, 3, 32
1, 3, 64
2, 3, 32
1, 3, 128
2, 3, 64
4, 3, 32
Cache hit[cj]
K1(1,128)
K1(1,256)
K1+2(1,128)
K1(1,512)
K1+2(1,256)
K1+2+4(1,128)
K1(2,64)
K1(2,128)
K1+2(2,64)
K1(2,256)
K1+2(2,128)
K1+2+4(2,64)
K1(3,32)
K1(3,64)
K1+2(3,32)
K1(3,128)
K1+2(3,64)
K1+2+4(3,32)
directed, 2kB, 16B/line
directed, 4kB, 16B/line
2-way, 4kB, 16B/line
directed, 8kB, 16B/line
2-way, 8kB, 16B/line
4-way, 8kB, 16B/line
directed, 2kB, 32B/line
directed, 4kB, 32B/line
2-way, 4kB, 32B/line
directed, 8kB, 32B/line
2-way, 8kB, 32B/line
4-way, 8kB, 32B/line
directed, 2kB, 64B/line
directed, 4kB, 64B/line
2-way, 4kB, 64B/line
directed, 8kB, 64B/line
2-way, 8kB, 64B/line
4-way, 8kB, 64B/line
Table 3: Configuration space for a single-level cache
The value of b depends on the word size (8-bytes) and the re-
spective line size in words. For example, a line size of 16-bytes
comprises 2 words (2 × 8-bytes). If 2b= 2-words, then b = 1.
Likewise, 32-bytes is b = 2, and 64-bytes is b = 3. Finally, the
number of sets (s) is determined by dividing the total size by the
line size and then by the associativity (w). In the configuration c18,
for example, 8-kBytes divided by 64-bytes gives us 128 lines, or 32
sets of 4 lines each (s = 32).
Since 64-bytes is the largest block size in the design space uti-
lized, it corresponds to 8 words or bmax = 3. The value of
smax is defined by the configuration with the maximum num-
ber of sets in the design space. It corresponds to configuration c4
in Table 3 (8-kByte directed-map cache with 16-bytes per line),
which give us smax = 512. Actually, the multi-layered table
limited by the bounds above (w = {1,2,4} × b = {1,2,3} ×
s = {32,64,128,256,512}) would be able to evaluate 45 differ-
ent cache configurations, of which the design space of 18 configu-
rations defined in the Table 3 is a subset of.
The right-hand column of the Table 3 shows how SPCE extracts
the number of hits from the multi-layered conflict table K, for ev-
ery cj in the design space. The number of cache misses can be
determined by subtracting the result from the trace size.
In order to validate our results for the whole space C, we deter-
mined cache miss rates for our suite of benchmarks with SPCE and
also with a very popular trace-driven cache simulator (DineroIV).
Then, we estimated the total energy consumed e(cj,ai) for run-
ning each benchmark aiwith the architecture configuration cj. We
adopted the same energy model as utilized by Zhang, so that our
results can be accurately compared with his state-of-the-art tuning
heuristic. The energy model calculates total energy consumption
for each cache configuration by calculating both static and dynamic
energy of the cache, main memory energy, and CPU stall energy.
We refer the reader to [19] for a detailed description of energy cal-
culation.
7.2Results
To validate SPCE, we compared the miss rates generated by
SPCE with the miss rates generated by DineroIV for the 45 cache
configurations supported by the multi-layered conflict table K. We
found the miss rate estimation to be identical. Table 4 shows sim-
ulation time speedups as high as 20.77 and an average of 14.88 for
cache evaluation. The variation in speedups for each application is
due to differences in locality.