Applications of Small-Scale Reconfigurability to Graphics Processors

Abstract

We explore the application of Small-Scale Reconfigurability (SSR) to graphics hardware. SSR is an architectural technique wherein functionality common to multiple subunits is reused
rather than replicated, yielding high-performance reconfigurable hardware with reduced area requirements. We show that SSR
can be used effectively in programmable graphics architectures to allow double-precision computation without affecting the
performance of single-precision calculations and to increase fragment shader performance with a minimal impact on chip area.

Full text

Page 1

Applications of Small-Scale Reconfigurability to
Graphics Processors
Kevin Dale1, Jeremy W. Sheaffer1, Vinu Vijay Kumar2, David P. Luebke1,
Greg Humphreys1, and Kevin Skadron1
1Department of Computer Science
2Department of Electrical and Computer Engineering
University of Virginia?
Abstract. We explore the application of Small-Scale Reconfigurability
(SSR) to graphics hardware. SSR is an architectural technique wherein
functionality common to multiple subunits is reused rather than repli-
cated, yielding high-performance reconfigurable hardware with reduced
area requirements. We show that SSR can be used effectively in pro-
grammable graphics architectures to allow double-precision computation
without affecting the performance of single-precision calculations and to
increase fragment shader performance with a minimal impact on chip
area.
1 Introduction
Every hardware system makes a tradeoff between performance and flexibility. At
one end of the spectrum, general purpose processors provide maximum flexibility
at the expense of performance, area, power consumption, and price. Custom
ASICs are the other extreme, providing maximum performance at a minimum
cost, albeit for only a very narrow set of applications.
Modern graphics hardware requires both high performance and flexibility,
placing it somewhere between these two extremes. Traditional intermediate hard-
ware solutions like FPGAs are inappropriate for graphics processors because of
their large size and low performance relative to their fixed-logic counterparts [1].
Small-scale reconfigurability (SSR) provides an attractive compromise; systems
that use SSR components can approach the high speed and small size of ASICs
while providing some specialized configurability. In this paper, we explore the
applicability of SSR to programmable graphics hardware.
The simplest example of a reconfigurable component is two fully functional
components connected with a multiplexer (see Fig. 1). Although these two com-
ponents are disjoint, in typical usage they will contain substantially similar re-
dundant substructures, which is precisely the situation in which SSR performs
best. Rather than replicate all of the redundant structure, one can instead reuse
common substructure within a single component.
A common SSR unit is the morphable multiplier. These multiplier-adders can
be reconfigured into a multiplier or an adder in a single cycle. When used to
?{kdale, jws9c, vv6v, luebke, humper, skadron}@virginia.edu
K. Bertels, J.M.P. Cardoso, and S. Vassiliadis (Eds.): ARC 2006, LNCS 3985, pp. 99–108, 2006.
c ? Springer-Verlag Berlin Heidelberg 2006

Page 2

100K. Dale et al.
AB
CD
AB
EF
MUX
(a) Na¨ ıve reconfigurable hardware
AB
CDEF
MUX
(b) A more efficient solution
Fig.1. A na¨ ıve implementation of reconfigurable hardware can be built by simply
multiplexing between two distinct, unmodified units (a), but a more efficient design
would reuse common substructure to avoid replication (b)
create single-precision floating point units, morphable multipliers yield a nearly
17% reduction in total area when compared to the sum of the sizes of their
constituent parts [2].
Graphics processors, like specialized multimedia processors and DSPs, are a
particularly suitable target for SSR due to their vector-processor like opera-
tions. When the same operation is performed repeatedly in SIMD fashion, re-
configuration and its associated overhead is infrequently needed, and any cost
can be amortized over many instructions. Furthermore, SSR-based components
typically have lower static power requirements because less hardware goes un-
used.
2Related Work
Dynamically reconfigurable hardware has been a popular topic in recent com-
puter architecture literature, especially in the FPGA and reconfigurable com-
puting communities. The configurability of these systems serves myriad design
goals, among them improved performance, power, area, and fault tolerance char-
acteristics.
Even et al. describe a dual mode IEEE multiplier—a pipelined unit capable
of producing one double-precision or two single-precision multiplications every
clock cycle with a three cycle latency [3]. The authors argue that the reuse of
substructure yields a cheap device that performs well for both precisions. They
further claim that the single precision mode is particularly useful for SIMD
applications, like graphics, because it is conducive to systems on which the same
operation is regularly repeated on large numbers of data points.

Page 3

Applications of Small-Scale Reconfigurability to Graphics Processors101
Guerra et al. explore built-in-self-repair (BISR) and its application to fault
tolerance, manufacturability, and application-specific programmable processor
design [4]. Previous work in the area of dynamic repair had made use of spe-
cialized redundant units to replace damaged units; their paper describes the
synthesis of more general units that can replace any of several units on a chip
when damage is detected. The authors coin the term HBISR (heterogeneous
BISR) for the technique.
A morphable multiplier is a device capable of performing either a floating point
multiply or add using the same hardware structure [2]. Morphable multipliers
require less area than the sum of the area needed for a separate multiplier and
adder (in fact, they require only slightly more than a multiplier alone), while
imposing negligible performance penalties.
Metrics like area, performance, and power are easily quantified, but it is less
obvious how to measure the increasingly important metric of hardware flexibility.
Compton and Hauck have defined a testing method and quantification metric
for flexibility of reconfigurable hardware [5]. Other examples of relevant research
in reconfigurable hardware include Kim et al. [6] and Chiou et al. [7].
The work in this paper makes use of Brook [8], a stream-based program-
ming language which allows the programmer to write general-purpose applica-
tions for a GPU without worrying about the sometimes byzantine details of
GPU programming. Our experiments all use Chromium [9] to intercept and an-
alyze streams of graphics commands made by real applications. The primary
advantage of using Chromium is that we ensure that our workloads are not con-
trived. Although we use Brook and Chromium without modification, we have
enhanced the Qsilver graphics architectural simulator [10, 11] to model the nec-
essary aspects of the fragment pipeline. A detailed description of our modi-
fications to QSilver and our experimental setup are presented in Sect. 3 and
Sect. 4.
3Simulation Setup
Qsilver is a simulation framework for graphics architectures that can simulate
low-level GPU activity for any existing OpenGL application [10]. Qsilver uses
Chromium [9] to intercept and transform an OpenGL application’s API calls
and create an annotated trace that encapsulates geometry, timing, and state
information. This trace serves as input to the Qsilver simulator core, which
performs an accurate timing simulation of the graphics hardware and produces
detailed statistics.
Qsilver is configured at runtime with a description of its pipeline. In these
experiments we simulate an NV4x-like architecture, with a pipeline configuration
similar to that of NVIDIA’s 6800 GT, so we configure Qsilver to model a system
with 6 vertex pipelines and 16 fragment pipelines. The fragments are tiled in
blocks of 2×2, so we effectively have 4 tile pipelines, each of which can process
4 fragments simultaneously. NV4x GPUs use a similar tiled configuration in the
fragment engine [12].

Page 4

102K. Dale et al.
To account for modifications to the fragment pipeline, we enhanced Qsil-
ver to track fragment shader activity. Our modified Qsilver simulator stores a
per-triangle identifier which uniquely specifies which, if any, fragment shader
was bound when that triangle was being rendered. We also store the text of
the fragment shaders so that they can be analyzed by the Qsilver simulator
core.
Both of the following experiments hold fixed the graphics pipeline described
above and focus on the programmable path of the fragment engine. While the
NV4x vertex engine follows a MIMD architecture, its fragment engine is truly
SIMD in nature. Additionally, in many modern games the majority of fragments
are shaded by fragment programs (see Fig. 5), so we focus our efforts on the
programmable path in the fragment engine.
Fragment processor
Stage 2
Stage 1 (+ Texture)
Branch Processor
Rasterizer
(a) Fragment processor
Crossbar
MUL MUL MUL MUL
Texture Operations
SFU
(b) Stage 1
ADDADD ADD ADD
Crossbar
Crossbar
SFU
MULMULMULMUL
Crossbar
(c) Stage 2
Fig.2. Baseline fragment units used for comparison. Stage 2 can take up to three
4-channel operands, one of which directly feeds the ADD units and whose data path
is represented here by dashed lines. Note the additional data paths that cascade the
ADD units; these allow for a single-pass dot product [13].
Our baseline fragment pipeline, depicted in Fig. 2, is similar to that found in
NV4x GPUs1. A single fragment unit contains two stages; four-channel fragments
(RGBA) reach stage 1 from either the rasterizer or fragment pipeline loopback.
Stage 2 can execute instructions in parallel with stage 1 in dual-issue mode
as well sequentially, taking its operands from the output of stage 1. Crossbars
route operands to the appropriate functional units, and Special Function Units
(SFUs) are used to perform special scalar operations like reciprocal square root.
The fragment units can also operate in co-issue mode, whereby a single 4-channel
data path functions as two distinct data paths, with independent instructions
executing in parallel, on the same unit, across these two data paths—e.g., a
3-vector and a scalar, or two 2-vectors [12].
1Based on those details that have been made available to the public or indirectly
obtained via patents and extensive benchmark tests. For additional details, see [12]
and [13].

Page 5

Applications of Small-Scale Reconfigurability to Graphics Processors103
4Experiments and Results
In this section, we describe two experiments we performed to validate our hy-
pothesis that using SSR components in a modern GPU architecture can ben-
efit certain applications. We show improved performance in the recent game
Doom III with only a minimal impact on GPU die area and also demonstrate
that double-precision floating point capabilities can be added to the fragment
pipeline without affecting the performance of single-precision applications.
Crossbar
FACFACFAC FAC
Texture Operations
SFU
(a) Stage 1
FAC FACFACFAC
Crossbar
Crossbar
SFU
FACFACFACFAC
Crossbar
(b) Stage 2
Fig.3. Proposed SSR fragment units for the first experiment. FAC modules are Flexible
Arithmetic Units, and they replace each of the ADD and MUL units in our baseline
architecture.
4.1 Increased Throughput
We first compared the simulated performance of the NV4x-like fragment pipeline
to that of an SSR fragment pipeline architecture, whose fragment units are
depicted in Fig. 3. The fragment units in our target SSR architecture are similar
to those in the baseline architecture; however,we replace both the multipliers and
adders in stages 1 and 2 with single-precision Flexible Arithmetic Units (FACs).
An FAC can be very quickly reconfigured to perform either a multiplication or
an addition and uses only slightly more gates than a multiplier. With current
technology, these FACs can produce a result every cycle and can be reconfigured
between cycles, assuming a 400 MHz clock and a two-stage pipeline [2]. Finally,
in the first set of FACs in our SSR architecture, we duplicate the accumulate
data paths from the baseline architecture’s ADD units. These data paths require
a trivial amount of additional area overhead.
In addition to supporting all the existing functionality of our baseline units,
the modified SSR units provide new scheduling opportunities beyond those of the
baseline. First, the baseline fragment pipe is only capable of performing a single
full-precision 4-vector addition per pass in stage 2 [13], while the SSR pipeline
is capable of performing three in one pass—one in stage 1 and two chained
additions in stage 2 (see Fig. 4a). Moreover, there is more freedom to schedule

Page 6

Stage 2
Stage 1 (+ Texture)
FAC x 4 (1)
FAC x 4 (3)
FAC x 4 (2)
+
+
+
Stage 2
Stage 1 (+ Texture)
FAC x 4 (1)
FAC x 4 (3)
FAC x 4 (2)
x
x
+
104K. Dale et al.
dot product and multiply-accumulate operations, both of which are extremely
common in fragment programs. For example, the SSR pipeline can execute a 32-
bit 3-channel dot product (DP3) and dependent scalar-vector multiplication—
e.g., the expression (A · B)C—in a single pass by computing the per-channel
multiply of A and B in stage 1, accumulating the channel products to obtain
A · B in the first set of FACs in stage 2, and performing a scalar-vector multiply
in its second set of FACs (Fig. 4b). Extending this scheduling approach to co-
issue configurations is straightforward.
(a) Single-pass
multiple 4-vector additions.
configuration for(b) Single-pass
compute (A · B)C.
configurationto
Fig.4. Two example configurations that provide additional scheduling opportunities
for the SSR fragment pipeline
Given these additional scheduling opportunities and the known scheduling
constraints of NV4x GPUs, we hand-scheduled fragment programs intercepted
from a 50-frame Doom III demo (see Fig. 5), which was then simulated un-
der Qsilver. We used NVShaderPerf2—a utility that displays shader scheduling
information for NVIDIA hardware—to schedule programs for our baseline ar-
chitecture simulation. While NV4x GPUs have dedicated hardware for perform-
ing common half-precision operations in parallel with full-precision operations,
none of the fragment programs tested included any half-precision operations.
However, to be sure of a legitimate comparison of performance along the full-
precision path, we forced NVShaderPerf to schedule programs for our NV4x-like
architecture using the full-precision path only. We limited program schedules for
the SSR architecture to the full-precision path as well.
From the simulation of this data stream, we obtained a 4.27% speedup over the
entire graphics pipeline for the SSR architecture. Equally as important, based on
conservative inverter-equivalent gate count estimates3, each FAC requires 12,338
2Unified compiler version 77.80.
3All area estimates are given in terms of inverter-equivalent gate area unless otherwise
specified.

Page 7

Applications of Small-Scale Reconfigurability to Graphics Processors 105
(a)(b)
Fig.5. Screen captures from Doom III. On the left, the color of each pixel is modulated
to indicate which fragment program generated it. The right image is the unmodified
rendering from the game. Notice that the majority of pixels are generated by program-
mable fragment shaders.
gates, only 710 more than a single-precision multiplier (11,628 gates). Replac-
ing the adders (7,782 gates) requires 4,556 additional gates. This additionally
requires the small overhead of a multiplexer to configure the FACs. Given these
gate estimates, with 16 fragment pipelines, the cost of our proposed use of SSR
is 382,464 gates, which is less than 0.2% of the total area of NVIDIA’s 6800 GT
(an estimated 222 million transistors[14]).
4.2Dual-Mode IEEE Adders and Multipliers
The GPGPU and scientific computing communities would like to have the ability
to perform double-precision calculations on the GPU. Unfortunately for them,
the gaming industry drives the graphics hardware industry, and games do not
currently require double-precision. We present a method here that can satisfy
the demands of the scientific community without compromising the performance
of the single-precision path so crucial to video game performance.
A dual-mode floating point unit is a small-scale reconfigurable unit capable of
performing two simultaneous single-precision operations or one double-precision
operation. Dual-mode units can be fully pipelined to produce results every cycle.
Like other SSR units, dual-mode multipliers and adders require internal multi-
plexers for path selection. Additionally, they require a rounding unit capable of
flexible rounding modes. The total additional structure for this modification is
insignificant [3].
We simulate a pipeline in Qsilver that uses dual-mode multipliers and adders
in the fragment engine, where we replace pairs of single-precision FPUs in the
baseline architecture with a single corresponding dual-mode FPU. This effec-
tively gives us an 8-wide double-precision fragment engine with approximately
half the throughput of the single-precision configuration. Double-precision con-
figuration also requires that we retask pairs of 32-bit registers as single 64-bit

Page 8

106K. Dale et al.
registers. With half as many fragment pipelines, each double-precision pipe has
the same number of available 64-bit registers as each single-precision pipe has 32-
bit registers (four 32-bit registers per fragment in the case of NV4x GPUs [12]).
By a similar argument, the bandwidth requirements for the memory and regis-
ter bus systems in 8-wide double-precision mode should not exceed those of the
original 16-wide single-precision configuration.
We have conservative area estimates for a double-precision adder and multi-
plier of 13,456 gates and 37,056 gates, respectively. The real overhead here comes
from replacing each pair of single-precision FPUs with one dual-mode FPU, at
an approximate cost of 815,744 gates over the entire fragment engine, or 0.4%
of the 6800 GT’s total area. Note that we have modified only the multiplica-
tion and addition units, so additional precision is not available for specialized
operations such as logarithms or square roots. Although many scientific ap-
plications would benefit greatly from high precision addition and multiplication
alone, a full double-precision arithmetic engine would be ideal. Dual-mode recip-
rocal, square-root, logarithm, and other specialized units are a topic for future
exploration.
To validate our SSR-based graphics architecture capable of both single- and
double-precision, we traced four Brook demo programs through Qsilver:
1. bitonic sort, a parallel sorting network
2. image proc(25,25), an image convolution shader
3. particle cloth(5,10,15), a cloth simulation
4. volume division(100), a volume isosurface extractor.
The results are summarized in Table 1. This table lists the cycle counts for each
application in both single- and double-precision modes. Note that the double-
precision calculations never require more than twice as long as the corresponding
single-precision calculation. Because the timing results are identical for dual-
mode units configured in single-precision mode and dedicated single-precision
units, we have shown that by using SSR we can add double-precision addition and
multiplication to the graphics pipeline with only a modest increase in gate count
and without affecting the performance of the commonly-used single-precision
path.
5Conclusions
We have extended Qsilver to record information on fragment program state in
its annotated trace. Our modified Qsilver core then uses this new information,
along with fragment program listings and timing information, to model the pro-
grammable fragment engine of an NV4x-like architecture. With this framework
in place, we have demonstrated the applicability of Small-Scale Reconfigurabil-
ity to graphics architectures. We have shown that it is possible to increase the
throughput of the fragment engine with only a small increase in die area. In ad-
dition, we have demonstrated that dual-mode multipliers and adders can provide
double-precision in the fragment engine to support scientific computing in the

Page 9

Applications of Small-Scale Reconfigurability to Graphics Processors107
Table 1. Single- and double-precision GPGPU computations using SSR. Each appli-
cation comes with the Brook distribution. The 32-bit cycles row shows the GPU cycle
count for our NV4x-like architecture. Note that these timings are identical whether we
are using a dual-mode unit configured in single-precision mode or a dedicated single-
precision unit. The 64-bit cycles row shows the cycles required for double-precision
after reconfiguration. As expected, none of the programs takes more than twice as long
with double-precision than with single-precision.
GPGPU community with no detriment to the gamers who drive the market. The
vector-like operations performed on GPUs make them a particularly good target
for such techniques, since need for reconfiguration is rare in SIMD environments,
and since the cost of reconfiguration is amortized over many operations.
6Future Work
The fragment engine is one of many elements of the graphics pipeline. Applica-
tions of SSR will likely yield similar performance improvements in other units
as well. Another area of exploration that is likely to be fruitful for SSR is power
consumption. Whenever portions of a chip are unused, they use no dynamic
power, but they leak static power. By their very nature, SSR components are
rarely idle, and should therefore leak a minimum of static power. Power leak-
age is currently a major issue with GPUs, and reducing leakage becomes crucial
as continuing improvements in chip manufacturing technology exacerbate this
problem [10].
Acknowledgments
We would like to thank John Lach for his input on SSR and Peter Djeu for his
collaboration on Chromium extensions. This work was funded by NSF grants
CCF-0429765, CCR-0306404, and CCF-0205324.
References
[1] Vijay Kumar, V., Lach, J.: Designing, scheduling, and allocating flexible arith-
metic components. In: Proceedings of the International Conference on Field
Programmable Logic and Applications. (2003)
[2] Chiricescu, S., Schuette, M., Glinton, R., Schmit, H.: Morphable multipliers. In:
Proceedings of the International Conference on Field Programmable Logic and
Applications. (2002)
Demo bitonic sort image proc particle cloth volume division
32-bit cycles
64-bit cycles
468
877
.534
1,292
2,525
.512
19,504
38,959
.501
254,923,418
509,846,783
.500
32-bit→64-bit speedup

Page 10

108 K. Dale et al.
[3] Even, G., Mueller, S.M., Seidel, P.M.: A dual mode IEEE multiplier. In: Pro-
ceedings of the International Conference on Innovative Systems in Silicon. (1997)
[4] Guerra, L.M., Potkonjak, M., Rabaey, J.M.: Behavioral-level synthesis of het-
erogeneous bisr reconfigurable asic’s. IEEE Transactions on VLSI (1998)
[5] Compton, K., Hauck, S.: Flexibility measurement of domain-specific reconfig-
urable hardware. In: Proceedings of the ACM/SIGDA Symposium on Field-
programmable Gate Arrays. (2004)
[6] Kim, K., Karri, R., Potkonjak, M.: Synthesis of application specific program-
mable processors. In: Proceedings of Design Automation. (1997)
[7] Chiou, L.Y., Bhunia, S., Roy, K.: Synthesis of application-specific highly effi-
cient multi-mode cores for embedded systems. ACM Transactions on Embedded
Computing Systems (2005)
[8] Buck, I., Foley, T., Horn, D., Sugerman, J., Fatahalian, K., Houston, M., , Han-
rahan, P.: Brook for GPUs: Stream computing on graphics hardware. ACM
Transactions on Graphics (2004)
[9] Humphreys, G., Houston, M., Ng, R., Ahern, S., Frank, R., Kirchner, P.,
Klosowski, J.T.:Chromium: A stream processing framework for interactive
graphics on clusters of workstations.
(2002) 693–702
[10] Sheaffer, J.W., Luebke, D.P., Skadron, K.: A flexible simulation framework for
graphics architectures. In: Proceedings of SIGGRAPH/Eurographics Workshop
on Graphics Hardware. (2004)
[11] Sheaffer, J.W., Skadron, K., Luebke, D.P.: Studying thermal management for
graphics-processor architectures. In: Proceedings of 2005 IEEE International
Symposium on Performance Analysis of Systems and Software. (2005)
[12] Kilgariff, E., Fernando, R. In: The GeForce 6 Series GPU Architecture. Addison-
Wesley Pub Co (2005) 471–491
[13] Seifert, A.: NV40 technology explained (2004) http://3dcenter.org/artikel/
nv40 pipeline/index3 e.php.
[14] Medvedev, A., Budankov, K.:NVIDIA GeForce 6800 Ultra (NV40) (2004)
http://www.digit-life.com/articles2/gffx/nv40-part1-a.html.
ACM Transactions on Graphics 21(3)