Uncovering hidden loop level parallelism in sequential applications

Abstract

As multicore systems become the dominant mainstream computing technology, one of the most difficult challenges the industry faces is the software. Applications with large amounts of explicit thread-level parallelism naturally scale performance with the number of cores, but single-threaded applications realize little to no gains with additional cores. One solution to this problem is automatic parallelization that frees the programmer from the difficult task of parallel programming and offers hope for handling the vast amount of legacy single-threaded software. There is a long history of automatic parallelization for scientific applications, but the techniques have generally failed in the context of general-purpose software. Thread-level speculation overcomes the problem of memory dependence analysis by speculating unlikely dependences that serialize execution. However, this approach has lead to only modest performance gains. In this paper, we take another look at exploiting loop-level parallelism in single-threaded applications. We show that substantial amounts of loop-level parallelism is available in general-purpose applications, but it lurks beneath the surface and is often obfuscated by a small number of data and control dependences. We adapt and extend several code transformations from the instruction-level and scientific parallelization communities to uncover the hidden parallelism. Our results show that 61% of the dynamic execution of studied benchmarks can be parallelized with our techniques compared to 27% using traditional thread-level speculation techniques, resulting in a speedup of 1.84 on a four core system compared to 1.41 without transformations.

Full text

Page 1

Uncovering Hidden Loop Level Parallelism in Sequential Applications
Hongtao Zhong, Mojtaba Mehrara, Steve Lieberman, and Scott Mahlke
Advanced Computer Architecture Laboratory
University of Michigan, Ann Arbor, MI 48109
{hongtaoz,mehrara,lieberm,mahlke}@umich.edu
Abstract
As multicore systems become the dominant mainstream
computing technology, one of the most difficult challenges
the industry faces is the software. Applications with large
amounts of explicit thread-level parallelism naturally scale
performance with the number of cores, but single-threaded
applications realize little to no gains with additional cores.
Onesolutionto this problem is automaticparallelizationthat
frees the programmer from the difficult task of parallel pro-
gramming and offers hope for handling the vast amount of
legacy single-threaded software. There is a long history of
automatic parallelization for scientific applications, but the
techniques have generally failed in the context of general-
purpose software. Thread-level speculation overcomes the
problem of memory dependence analysis by speculating un-
likely dependences that serialize execution. However, this
approachhas lead to only modest performance gains. In this
paper, we take another look at exploiting loop-level paral-
lelism in single-threaded applications. We show that sub-
stantial amounts of loop-level parallelism is available in
general-purpose applications, but it lurks beneath the sur-
face and is often obfuscated by a small number of data and
control dependences. We adapt and extend several code
transformations from the instruction-level and scientific par-
allelization communities to uncover the hidden parallelism.
Our results show that 61% of the dynamic execution of stud-
iedbenchmarkscanbeparallelizedwithourtechniquescom-
paredto 27% usingtraditionalthread-levelspeculationtech-
niques, resulting in a speedup of 1.84 on a four core system
compared to 1.41 without transformations.
1Introduction
Due to power dissipation and design complexity issues
of building faster uniprocessors, multicore systems have
emerged as the dominant architecture for mainstream com-
puter systems. Semiconductor companies now put two to
eightcoresona chip,andthis numberis expectedto continue
growing. One of the most difficult challenges going forward
is software: if the number of devices per chip continues to
growwith Moore’s law, can the available hardwareresources
beconvertedintomeaningfulapplicationperformancegains?
In some regards,the embeddedand domain-specificcommu-
nities have pulled ahead of the general-purposeworld in tak-
ing advantage of available parallelism, as most system-on-
chip designs have consisted of multiple processors and hard-
ware accelerators for some time. However, these system-
on-chip designs are often deployed for limited application
spaces, requiring hand-generated assembly and tedious pro-
gramming models. The lack of necessary compiler tech-
nology is increasingly apparent as the push to run general-
purpose software on multicore platforms is required.
In the scientific community, there is a long history of suc-
cessful parallelization efforts [1, 3, 7, 12, 15]. These tech-
niques target counted loops that manipulate array accesses
with affine indices, where memory dependence analysis can
be precisely performed. Loop-level and single-instruction
multiple-data parallelism are extracted to execute multiple
loop iterations or process multiple data items in parallel. Un-
fortunately, these techniques do not often translate well to
general-purpose applications. These applications are much
more complexthanthose typicallyseen in the scientific com-
puting domain, often utilizing pointers, recursive data struc-
tures, dynamic memory allocation, frequent branches, small
function bodies, and loops with small bodies and low trip
count.More sophisticated memory dependence analysis,
such as points-to analysis [23], can help, but parallelization
often fails due to a small number of unresolvable memory
accesses.
Explicit parallel programming is one potential solution to
the problem, but it is not a panacea. These systems may
burden the programmerwith implementation details and can
severely restrict productivity and creativity. In particular,
getting performance for a parallel application on a heteroge-
neous hardware platform, such as the Cell architecture, often
requires substantial tuning,a deepknowledgeof the underly-
ing hardware, and the use of special libraries. Further, there
is a large body of legacy sequential code that cannot be par-
allelized at the source level.
A well-researched direction for parallelizing general-
purpose applications is thread-level speculation (TLS). With
TLS, the architecture allows optimistic execution of code re-
gions before all values are known [2, 13, 14, 25, 30, 32, 34,
39]. Hardwarestructures track registerand memoryaccesses
to determine if any dependence violations occur. In such
cases, registerandmemorystate are rolled backto a previous
correct state and sequential re-execution is initiated. With
TLS, the programmer or compiler can delineate regions of
code believed to be independent[4, 9, 18, 20]. Profile data is
often utilized to identify regions of code that are likely inde-
pendent, and thus good candidates for TLS.
Previous work on TLS has yielded only modest perfor-
mance gains on general-purpose applications. The POSH
compiler is an excellent example where loop-based TLS
yielded approximately 1.2x for a 4-way CMP and loop com-
bined with subroutine TLS yielded approximately 1.3x on
SPECint2000 benchmarks [18]. That result improves upon
prior results reported for general-purposeapplications by the
Stampede and Hydra groups [13, 32]. One major limitation
of prior work is that parallelization is attempted on unmodi-
fied code generatedby the compiler. Real dependences(con-

Page 2

trol, register, or memory) often mask potential parallelism.
A simple example is the use of a scalar reduction variable
in a loop. All iterations update the reduction variable, hence
they cannot be run in parallel. One notable exception is the
work by Prabhu and Olukotun that looked at manually ex-
posing thread-level parallelism (TLP) in Spec2000 applica-
tions [26]. They examined manually transforming applica-
tions to expose more TLP, including introducing parallel re-
ductions. They showed substantial increases in TLP were
possible using a variety of transformations for traditional se-
quential applications. However, many of the transformations
were quite complex, requiring programmerinvolvement.
Our work is directly motivated by Prabhu and Olukotun.
We examine the feasibility of automatic compiler transfor-
mations to expose more TLP in general-purpose applica-
tions. We target automatic extraction of loop-level paral-
lelism, whereloopswithsets ofcompletelyindependentloop
iterations, or DOALL loops, are identified, transformed, and
parallelized. Memory dependence profiling is used to gather
statistics on memory dependence patterns in all loop bodies,
similar to prior work [18]. Note that we are not parallelizing
inherentlysequentialalgorithms. Rather, we focuson uncov-
ering hidden parallelism in implicitly parallel code.
We examine the use of TLS as a method to overcome
the limitations of static compiler analysis. This is the same
conclusion reached by prior work. However, we look be-
yond the nominal code generated by the compiler to find
parallel loops. We show that substantial loop-level paral-
lelism lurks below the surface, but it is obstructed by a va-
riety of control, register and memory dependences. To over-
come these dependences, we introduce a novel framework
and adapt and extend several code transformations from do-
mains of instruction-level parallelism and parallelization of
scientific codes. Specifically, our contributions are:
• DOALL loop code generation framework - We intro-
duce a novel framework for speculative partitioning of
chunkedloop iterations across multiple cores. The tem-
plate handles complex cases of uncountedloops as well
as counted ones and takes care of all scalar live-outs.
• TLP-enhancing code transformations - We design sev-
eralcodetransformationstobreakcrossiterationdepen-
dences in nearly DOALL loops. These transformations
arenotentirelynew,but ratherare variantsofpriortech-
niques that have different objectives and are adapted to
work in the presence of uncertain dependence informa-
tion and TLS hardware. The optimizations consist of of
speculative loop fission, speculative prematerialization,
and isolation of infrequent dependences.
2 Parallelization Challenges
To illustrate more concretely the challenges of identify-
ingTLPusingcompileranalysis ingeneral-purposecode,we
examine the frequency of provable DOALL loops in a vari-
ety of applications. We use the OpenImpact compiler sys-
tem that performs memory dependence analysis using inter-
procedural points-to analysis [23]. A total of 41 applications
(all C source code, or C code generated from f2c) from four
domains are investigated: SPECfp, SPECint, MediaBench,
and Unix utilities. The lower portion of the bars in Figure 1
show the ability of an advanced compiler to expose provable
DOALL parallelism. For each application, the fraction of se-
quential execution that is spent in provable DOALL loops is
presented. Toderivethisvalue,theexecutionfrequencyof all
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
052.alvinn
056.ear
171.swim
172.mgrid
177.mesa
179.art
183.equake
188.ammp
008.espresso
023.eqntott
026.compress
072.sc
099.go
124.m88ksim
129.compress
130.li
132.ijpeg
164.gzip
175.vpr
181.mcf
197.parser
256.bzip2
300.twolf
cjpeg
djpeg
epic
g721decodeg721encode
gsmdecode gsmencode
mpeg2dec mpeg2enc
pegwitdec pegwitenc
rawcaudio rawdaudio
unepic
grep
lex
yacc
average
SPEC FPSPEC INTMediabench Utilities
Fraction of sequential executio
Profiled DOALL
Provable DOALL
Figure 1: Fraction of sequential execution covered by prov-
ableDOALLloopsidentifiedthroughcompileranalysis(bot-
tom portionof bars) andspeculativeDOALL loopsidentified
through profiling (top portion of bars).
static operations that reside in at least one provable DOALL
loop are summed and divided by the total dynamic opera-
tion count. The figure shows that the compileranalysis is not
very successful with the exception of two cases, 171.swim
from SPECfp and mpeg2dec from MediaBench. The com-
piler is generallymost successful in SPECfp, where previous
scientific parallelizationtechniquesare applicable. However,
the pointer analysis is only partially successful at resolving
memory dependences in the non-scientific code.
TLS has the potential to provide large performance gains
by allowing speculative parallelization of loops where com-
pileranalysis aloneis unsuccessful. One key issue forTLS is
to parallelize the loops that have low probabilitymemory de-
pendences. Memory profiling is a way to estimate the mem-
ory dependences in a program. The memory profiler runs
the application on a sample input and records the memory
address accessed by every load and store. If two memory in-
structions access the same location, a memory dependenceis
recorded.
For the purpose of parallelizing loops, we only care about
cross iteration dependences. If the loops are nested, we need
to know in what nesting level each memory dependence is
happening. Furthermore, when the loop contains function
calls, we want to know if the called function accesses any
global variable which causes cross iteration memory depen-
dence. Therefore, we developed a control-aware memory
profiler to identify speculative DOALL loops.
The upper portion of the bars in Figure 1 show the frac-
tion of serial runtime spent in DOALL loops identified by
the profiler. A loop is speculative DOALL if it contains
zero or very few cross iteration memory dependences, and it
contains no cross iteration register and control dependences.
Note that simple register and control dependences, such as
those cause by induction variables, can easily be eliminated
and hence are ignored in this analysis. Comparing the pro-
filed and provable results, we see that many more DOALL
loopsareidentifiedusingprofileinformationthanusingcom-
piler memory analysis. On average, 28% of the sequential
execution is contained within speculative DOALLs, com-
pared with 8% in provable DOALLs. As expected, the pro-
filer is highly effective with the SPECfp applications, which
areknowntocontainlargeamountsofloop-levelparallelism.
However, for the remaining applications, the results are gen-
erally disappointing. With the exception of a few media ap-

Page 3

plications, most of these applications contain few DOALL
loops.
These poor results were confusing as we had observed
that many loops in these applications contained no sta-
tistically significant cross-iteration memory dependences.
Hence, if we were only looking at memory dependences, the
number of speculative DOALLs would be much larger. The
problem is that the loops are not DOALL due to other de-
pendences - namely cross iteration register and control de-
pendences. If these dependences could be broken by the
compiler, then the number of DOALL loops would increase
substantially. The remainderof the paper exploresthis direc-
tion of research, namely a set of compiler transformations to
break these dependences within the context of a speculative
execution environment.
3 Architectural Support
SpeculativeDOALL loops requireseveral underlyingfea-
tures to ensure correct execution and make recovery actions
when a speculation violation occurs. These features can be
implemented in hardware, software, or a combination of the
two. Traditional TLS hardware can support speculative ex-
ecution of all types of loops and acyclic code [30, 32, 13].
Since we are only executing speculative DOALL loops, a
subset of TLS hardware is required.
many of the loops studied have small bodies and low trip
counts, the overhead of parallel execution must be mini-
mized. In this section, we specify the underlying hardware
model and assumptions about its operation.
Our target architecture is a standard chip multiprocessor
with coherent L1 caches and a shared L2 cache. The sys-
tem is extendedwith three majorfeatures to supportspecula-
tiveDOALLexecution. First, atransactionalmemorysystem
similar to LogTM [22, 37] is utilized to detect memory de-
pendence violations and rollback execution if necessary. We
assume ordered transactions. In the case of a conflict, the
transaction with the higher ID is aborted. Larger transaction
IDs are assigned to higher groups of loop iterations to main-
tain sequential semantics. Since we only parallelize loops
with statistically zero dependences, memory values are not
forwarded from previous transactions to later transactions
to simplify the hardware implementation. If a later trans-
action uses values stored in a previous transaction, the later
transaction is aborted. Several new instructions are added to
the instruction set to expose the transactional memory to the
compiler. The XBEGIN instruction marks the beginningof a
transaction. XBEGIN takes the address of the abort handler
as an operand. The XCOMMIT instruction marks the end of
a transaction and commits the speculative state.
Second, a scalar operand network, similar to that in the
Raw architecture [33], is used to communicate register val-
ues between cores. A mechanism similar to register chan-
nels [11, 10] or a synchronization array [28] can also be
usedto supportregistercommunicationand synchronization.
Each core is extendedwith send and receive queues and sim-
ple routing logic (XY routing is assumed). Two new instruc-
tions, SEND and RECV, are added to the instruction set. The
SEND instruction has two source operands, a register and a
destination core ID. It reads the value in the source register
and sends it to the destination core. The RECV instruction
also takes two source operands, a target register and a sender
core ID. When a RECV is executed, it looks in the incoming
message queue in the core for messages from the sender ID.
If such a message is found, it moves the value to the target
Furthermore, since
register; otherwise, it stalls the core and waits for the mes-
sage.
SEND and RECV operations also provide an efficient
mechanism to guarantee the ordering of instructions in dif-
ferent cores. We commit loop iterations in original program
order to maintain sequential semantics by passing a commit
permission token between the cores using SEND/RECV in-
structions.
Finally, the hardware supports low latency thread spawn-
ing to efficiently exploit parallelism in small loops. We as-
sume theoperatingsystem pre-allocatesseveralcores to each
application. To spawn a new thread, a core simply sends a
program counter (PC) value to another core to initiate its ex-
ecution. Since the compilerexplicitly controlsthread spawn-
ing,thelive-inscalarvaluesfortheslaveareexplicitlypassed
from the master using the scalar operand network.
4 Uncovering Hidden Loop Level Parallelism
Asshownintheprevioussections,withoutanycodetrans-
formation, out-of-the-box loop level parallelization oppor-
tunities for general applications are limited, even with TLS
hardware support. After manually studying a wide range of
loops,we foundthatmanyparallelopportunitieswerehidden
beneath the seemingly sequential code. With proper code
transformations, critical cross iteration dependences can be
untangledresultingin manymorespeculativeDOALL loops.
In this section, wefirst introduceourcodegenerationscheme
for speculative DOALL loops. It handles both counted loop
anduncountedloopwith crossiterationcontroldependences.
Subsequently,wepresenttechniquestohandlecross iteration
dependencesthathinderlooplevelparallelism. Inadditionto
somewell-knowntechniques,weintroducethreenoveltrans-
formations to untangle register and memory dependences:
speculative loop fission, speculative prematerialization, and
isolation of infrequent dependences.
4.1Code Generation
After choosing candidate loops for parallelization us-
ing the profile information, the compiler distributes loop
execution across multiple cores.
gorize DOALL loops into DOALL-counted and DOALL-
uncounted. In DOALL-counted, the trip count is known
when the loop is invoked (note, the trip count is not nec-
essarily a compile-time constant). However, for DOALL-
uncounted, the trip count is unknown until the loop is fully
executed. While loops and for loops with conditional break
statements are two examples of DOALL-uncounted loops
that occur frequently. In these cases, the execution of every
iteration depends on the outcome of exit branches in previ-
ous iterations. Therefore, these loops contain cross iteration
control dependences. In this section, we introduce our code
generation framework for handling control dependences and
executing both speculative DOALL-counted and DOALL-
uncounted loops.
Figure 2 shows the detailed implementation of our code
generationframework. Inthe proposedscheme,theloopiter-
ations are divided into chunks. The operating system passes
the numberof availablecores and the chunksize to the appli-
cation. Our framework is flexible enough to use any number
of available cores forloop execution. We insert an outerloop
around the original loop body to manage the parallel execu-
tion between different chunks. Following is a description of
the functionality of each segment in Figure 2.
In this work, we cate-

Page 4

RECV live _outs and
last_upd_idx values
Set live_outj to the
last updated value
for (all live_outs)
SEND(live_outj,0)
SEND(last_upd_idxj,0)
DONE
Receive CS , SS, TC and live -ins
IS = IS_init + CS * SS * thread_id
for (all live_outs)
last_upd_idxi = -1;
THREAD0?
Initialization
Parallel
Loop
Consolidation
Abort
Handler
for (i=IS;i<IE;i+=SS)
// original loop code
live_outk =… //kth liveout assignment
last_upd_idxk= i;
if (brk_cond)
local_brk_flag = 1;
break;
XBEGIN
if (global_brk_flag) break;
IE = min(IS+CS*SS,n)
perm = RECV(THREADj-1)
XCOMMIT
if (local_brk_flag)
global _brk_flag = 1;
kill_other_threads;
elseif (IE < n)
SEND(perm,THREADj+1)
IS+=CS * TC * SS;
if (IS < n) &
(!local_brk_flag)
Spawn jobs to available cores
Send CS, SS, TC and live -ins to all threads
Spawn
Yes
No
Figure 2:
ulative loop parallelization. Transaction scope is marked
by XBEGIN and XCOMMIT. (CS: chunk size, IS: iteration
start, IE: iteration end, SS: step size, TC: thread count)
Detailed code generation framework for spec-
Spawn: To start, the master thread spawns threads con-
taining chunks of loop iterations. It sends the necessary pa-
rameters (chunksize, thread count, etc.) and live-in values to
all threads.
Initialization: In the initialization block, all participat-
ing cores receive the required parameters and live-in values.
Since live-in values are not changed in the loop1, we only
send them once for each loop execution. This block also
computes the starting iteration, IS, for the first chunk. After
initialization,each core managesits own iteration start value,
thus parallel execution continues with minimum interaction
among the threads.
In order to capture the correct live-out registers after par-
allel loop execution, we use a set of registers called last-upd-
idx, one for each conditional live-out (i.e., updated in an if-
statement). When a conditional live-out register is updated,
we set the corresponding last-upd-idx to the current itera-
tion number to keep track of the latest modifications to the
live-out values. If the live-out register is unconditional (i.e.,
updated in every iteration), the final live-out value can be re-
trieved from the last iteration and no last-upd-idx is needed.
Abort Handling: The abort handler is called when a
transaction aborts. If the TM hardware does not backup the
register file, we can use the abort handler to recover certain
registervaluesin caseof transactionabort. Morespecifically,
we need to recover the live-out and last-upd-idx register val-
ues. We need to backup these registers at the beginning of
1If a live-in value is changed in the loop, it generates a cross iteration
register dependence and the loop cannot be parallelized.
each transaction, and move the backup value to the registers
in the abort handler. Moreover, we also add recovery code
in the abort handler for some of our transformations as de-
scribed in the next section.
Parallel Loop: The program stays in the parallel loop
segment as long as there are some iterations to run and no
break has happened. In this segment, each thread executes
a set of chunks. Each chunk runs iterations from IS to IE.
The value of IS and IE are updated after each chunk using
the chunk size (CS), thread count (TC) and step size (SS).
Each chunk of iterations are enclosed in a transaction, de-
marcated by XBEGIN and XCOMMIT instructions. The
transactional memory monitors the memory accessed by
each thread. It aborts the transaction running higher itera-
tions ifa conflictis detected,andrestarts thetransaction from
the abort handler. Chunks are forced to commit in-order to
maintain correct execution and enable partial loop rollback
and recovery. In orderto minimize the required bookkeeping
for this task, we use a distributed commit ordering technique
in which each core sends commit permission to the next core
after it commits successfully. These permissions are sent in
the form of dummy values to the next core. The RECV
command near the end of the main block causes the core to
stall and wait for dummy values from the previous core.
For uncounted loops, if a break happens in any thread,
we don’t want to abort higher transactions immediately, be-
cause the execution of the thread is speculative and the break
could be a false break. Therefore, we use a local variable lo-
cal brk flag in each thread to keep track of if a chunkbreaks.
If the transaction commits successfully with local brk flag
set, the break is not speculative any more, and a transaction
abort signal is sent to all threads using a software or hard-
ware interrupt mechanism. In addition, a global brk flag is
set, so that all threads break the outer loop after restarting
the transaction as a result of the abort signal. The reason for
explicitly aborting higher iterations is that if an iteration is
started by misspeculation after the loop breaks, it could pro-
duce an illegal state. The execution of this iteration might
cause unwanted exceptions or might never finish if it con-
tains inner loops.
Consolidation: After all cores are done with the execu-
tion of iteration chunks, they enter the consolidation phase.
In this period, each core sends its live-outs and last-upd-idx
array to THREAD0that selects the last updated live-out
values. All threads are terminated after consolidation and
THREAD0continues with the rest of program execution.
We carefully designed the framework to keep most of the
extra code outside the loop body, so they only execute once
per chunk . The overhead in terms of total dynamic instruc-
tions is quite small.
4.2 Dependence Breaking Optimizations
This section focuses on breaking cross iteration regis-
ter dependences that occur when a scalar variable is de-
fined in one iteration and used in another. First, we exam-
ine several traditional techniques that are commonly used
by parallelizing compilers for scientific applications: vari-
able privatization, reduction variable expansion, and ignor-
inglongdistancememorydependences. These optimizations
are adapted to a speculative environment. Then, we pro-
pose three optimizations specifically designed for a specu-
lative environment: speculative loop fission, speculative pre-
materialization and infrequent dependence isolation. These
transformations are adaptations of existing ones used in the

Page 5

1: while (node) {
2: work(node);
3: node = node->next;
}
// Sequential
count = 0;
1: while (node) {
4: node_array[count++]= node;
3: node = node->next;
}
// Parallel
XBEGIN
node = node_array[IS];
i = 0;
1': while (node && i++ < CS) {
2: work(node)
3': node = node->next
}
if (node != node_array[IS+CS]
kill_higher_iter_THREADs();
XCOMMIT
(a)
(b)
(c)
rf
rf
rf
1
2
3
c
c
m?
rf
rf
rf
1
4
3
c
c
rf
rf
rf
1
2
3'
c
c
m?
rf
rf
rf
1'
2
3'
c
c
m?
Sequential
Parallel
...
(d)
Figure 3: Speculative loop fission (a) Original loop (b) Original data flow graph (c) Data flow graph after fission (d) Loop after
fission - (rf: register flow dependence, c: control dependence, m?: unlikely cross-iteration memory dependence
scientific and ILP compilers.
4.2.1 Traditional Dependence Breaking Optimizations
Variable privatization. Cross iteration input, anti- and out-
put dependences can be removed by register privatization.
Since each core has a separate register file, register accesses
in different cores are naturally privatized. Live-in scalars are
broadcast to each core during initialization, thus all false de-
pendences on scalars are removed. Handling output depen-
dences for live-out variables is tricky. Since the value of a
live-out variable is used outside the loop, we need to find out
whichcoreperformedthelast writetotheregister. Thisis not
obvious if the register is updated conditionally. As described
in the previoussection, the code generationtemplate handles
this case by assigning an integer value on each core for each
live-out register. This value is set to the last iteration index
where the live-out variable is written. The compiler inserts
code after the loop (in the consolidation block in Figure 2)
to set the live-out registers to their last updated values in the
loop based on the stored iteration index.
Reduction variable expansion.
such as accumulatorsorvariables that are used to find a max-
imum or minimum value, cause cross iteration flow depen-
dences. The most common case is the sum variable when all
elements of an array are summed. These dependencescan be
removed by creating a local accumulator (or min/max vari-
able) for each core, and privately accumulating the totals on
each core. After the loop and in the consolidation block, lo-
cal accumulatorsare summed orthe global min/maxis found
amongst the local min/max’s.
Ignoring long distance memory dependences. When
the number of iterations between two memory depen-
dences is larger than some threshold, there is an oppor-
tunity for parallelization by simply ignoring the depen-
dence. Intuitively, if the distance between memory ac-
cesses is n, the compiler can make n − 1 iterations exe-
cute in parallel. Subsequently, if we set the chunk size as
cross iteration distance/number of cores, our scheme
Reduction variables,
in Section 4.1 generates the proper code for parallel execu-
tion of the loop.
4.2.2 Speculative Loop Fission
By studying benchmarks, we observed that many loops con-
tain large amounts of parallel computation, but they cannot
be parallelized due to a few instructions that form cross iter-
ation dependencecycles. We call these loops almost DOALL
loopas thebulkoftheloopis DOALL,butasmallrecurrence
cycle(s) inhibits parallelization. The objective of speculative
loop fission is to split the almost DOALL into two parts: a
sequential portionthat is run on one core followed by a spec-
ulative DOALL loop that can be run on multiple cores. The
basic principles of this optimization are derived from tradi-
tional loop fission or distribution [1].
Figure 3(a) shows a classic example of such a loop. A
linked list is iterated through, with each iteration doing some
work on the current node. Figure 3(b) shows the data de-
pendence graph for the loop, with the important recurrence
of operation 3 to itself. Note that there may be an unlikely
memorydependencebetweenoperations2 and3 as indicated
by the “m?” edge in the graph. Such a situation occurs when
the compiler analysis cannot prove that the linked list is un-
modified by the work function. For this loop, the sequential
portionconsists of the pointerchasingportion(i.e., operation
3)andtheDOALLportionconsistsoftheworkperformedon
each node (i.e., operation 2).
The basic transformation is illustrated in Figure 3(c). The
strongly connected components, or SCCs, are first identi-
fied to compose the sequential portion of the loop. Depen-
dences that will be subsequently eliminated are ignored dur-
ing this process. These include control dependences handled
by the DOALL-uncounted schema (i.e., the control depen-
dence from operation 1 to 3), unlikely memory dependences
(i.e., the memory dependence from operation 2 to 3), and
register dependences caused by reduction variables. In this
example, the SCC is operation 3. The sequential portion is
then populated with two sets of nodes. First, copies of all