Accelerator validation of an FPGA SEU simulator
ABSTRACT An accelerator test was used to validate the performance of an FPGA single event upset (SEU) simulator. The Crocker Nuclear Laboratory cyclotron proton accelerator was used to irradiate the SLAAC1-V, a Xilinx Virtex FPGA board. We also used the SLAAC1-V as the platform for a configuration bitstream SEU simulator. The simulator was used to probe the "sensitive bits" in various logic designs. The objective of the accelerator experiment was to characterize the simulator's ability to predict the behavior of a test design in the proton beam during a dynamic test. The test utilized protons at 63.3 MeV, well above the saturation cross-section for the Virtex part. Protons were chosen because, due to their lower interaction rate, we can achieve the desired upset rate of about one configuration bitstream upset per second. The design output errors and configuration upsets were recorded during the experiment and compared to results from the simulator. In summary, for an extensively tested design, the simulator predicted 97% of the output errors observed during radiation testing. The SEU simulator can now be used with confidence to quickly and affordably examine logic designs to 'map' sensitive bits, to provide assurance that incorporated mitigation techniques perform correctly, and to evaluate the costs and benefits of various mitigation strategies. The simulator provides an excellent test environment that accurately represents radiation induced configuration bitstream upsets.
Article: A Space-Based Reconfigurable Radio[show abstract] [hide abstract]
ABSTRACT: Field Programmable Gate Arrays (FPGAs) offer substantial improvements in processing throughput over microprocessors. This paper presents the design for a remote sensing payload using radiation tolerant FPGAs to process two 40Mhz channels of the radio spectrum. The system design mitigates the effects of single event upsets in the FPGAs. The mechanical design enhances system reliability, which is stressed due to substantial thermal loads produced by the devices. System performance and test results will also be discussed.
- [show abstract] [hide abstract]
ABSTRACT: The performance, in-system reprogrammability, flexibility, and reduced costs of SRAM-based field programmable gate arrays (FPGAs) make them very interesting for high-speed on-orbit data processing, but the current generation of radiation-tolerant SRAM-based FPGAs are based on commercial-off-the-shelf technologies and, consequently, are susceptible to single-event upset effects. In this paper, we discuss in detail the consequences of radiation-induced single-event upsets (SEUs) in the state of half-latch structures found in Xilinx Virtex FPGAs and describe methods for mitigating the effects of half-latch SEUs. One mitigation method's effectiveness is then illustrated through experimental data gathered through proton accelerator testing at Crocker Nuclear Laboratory, University of California-Davis. For the specific design and mitigation methodology tested, the mitigated design demonstrated more than an order of magnitude improvement in reliability over the unmitigated version of the design in regards to average proton fluence until circuit failure.IEEE Transactions on Nuclear Science 01/2004; · 1.22 Impact Factor
- Aerospace Conference, 2003. Proceedings. 2003 IEEE; 02/2003
Approved for public release;
distribution is unlimited.
Form 836 (8/00)
Los Alamos National Laboratory, an affirmative action/equal opportunity employer, is operated by the University of California for the U.S.
Department of Energy under contract W-7405-ENG-36. By acceptance of this article, the publisher recognizes that the U.S. Government
retains a nonexclusive, royalty-free license to publish or reproduce the published form of this contribution, or to allow others to do so, for U.S.
Government purposes. Los Alamos National Laboratory requests that the publisher identify this article as work performed under the
auspices of the U.S. Department of Energy. Los Alamos National Laboratory strongly supports academic freedom and a researcher’s right to
publish; as an institution, however, the Laboratory does not endorse the viewpoint of a publication or guarantee its technical correctness.
Accelerator Validation of an FPGA SEU Simulator
D. Eric Johnson, LANL, BYU
Michael Caffrey, LANL
Paul Graham, LANL
Nathan Rollins, BYU
Michael Wirthlin, BYU
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 50,
NO. 6, DECEMBER 2003
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 50, NO. 6, DECEMBER 20031
Accelerator Validation of an FPGA SEU Simulator
Eric Johnson, Michael Caffrey, Paul Graham, Nathan Rollins, Michael Wirthlin
Abstract—An accelerator test was used to validate the per-
formance of an FPGA Single Event Upset (SEU) simulator. The
Crocker Nuclear Laboratory cyclotron proton accelerator was
used to irradiate the SLAAC1-V, a Xilinx Virtex FPGA board.
We also used the SLAAC1-V as the platform for a configuration
bitstream SEU simulator. The simulator was used to probe the
“sensitive bits” in various logic designs. The objective of the
accelerator experiment was to characterize the simulator’s ability
to predict the behavior of a test design in the proton beam during
a dynamic test. The test utilized protons at 63.3 MeV, well above
the saturation cross-section for the Virtex part. Protons were cho-
sen because, due to their lower interaction rate, we can achieve
the desired upset rate of about one configuration bitstream
upset per second. The design output errors and configuration
upsets were recorded during the experiment and compared to
results from the simulator. In summary, for an extensively tested
design, the simulator predicted 97% of the output errors observed
during radiation testing. The SEU simulator can now be used
with confidence to quickly and affordably examine logic designs
to ‘map’ sensitive bits, to provide assurance that incorporated
mitigation techniques perform correctly, and to evaluate the
costs and benefits of various mitigation strategies. The simulator
provides an excellent test environment that accurately represents
radiation induced configuration bitstream upsets.
Index Terms—SEU, FPGA, simulator, proton accelerator, ra-
diation, dynamic testing
based applications such as remote sensing. FPGAs are pro-
grammable logic devices, which allow the user to specify the
function to be performed. There are many available resources
within an FPGA to perform various logic functions. The
way in which these resources are utilized and interconnected
is specified by the circuit design, also known as a con-
figuration bitstream. The configuration bitstream determines
which resources within the FPGA are used to implement a
specific logic design. The configuration is sensitive to SEUs,
some of which will result in changes to the design. This
concept is illustrated in Figure 1. SRAM based FPGAs can be
reprogrammed quickly (??1 sec) and indefinitely with new
configuration bitstreams. This feature is exploited extensively
in this work for error introduction. It is also a compelling
capability for deployed systems.
Reconfigurable FPGAs within a spacecraft allow the use
of digital circuits that are both application-specific and re-
programmable. Unlike application-specific integrated circuits
HERE is increasing interest in the use of SRAM-based
Field Programmable Gate Arrays (FPGAs) in space-
Manuscript received August 29, 2003. This work was supported by the
Department of Energy at Los Alamos National Laboratory.
P. Graham and M. Caffrey are with Los Alamos National Laboratory, Los
D. E. Johnson is with both Los Alamos National Laboratory and Brigham
N. Rollins and M. Wirthlin are with the Department of Electrical and
Computer Engineering, Brigham Young University, Provo, UT.
Fig. 1. a) Example FPGA architecture. b) The configuration bitstream defines
device function. c) An example circuit
(ASICs), FPGAs can be configured after the spacecraft has
been launched. This flexibility allows the same FPGA re-
sources to be used for multiple instruments, missions, or
changing spacecraft objectives. Errors in an FPGA design can
be resolved by fixing the incorrect design and reconfiguring
the FPGA with an updated configuration bitstream.
While the use of reprogrammable FPGAs for spaced-based
applications offers a number of important advantages, these
SRAM-based FPGAs are very sensitive to both heavy ion
and proton induced single event upsets (SEUs). Such up-
sets affect all types of internal FPGA state including user
design flip-flops, the FPGA configuration bitstream, and half-
latches,. Upsets in the FPGA configuration bitstream are
especially problematic because they can change the actual
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 50, NO. 6, DECEMBER 20032
circuit design. The configuration bitstream represents the bulk
of the state in the device, so understanding the behavior of the
device in the presence of SEUs is important.
A configuration bitstream SEU simulator was created to
study the impact of upsets within the configuration memory on
FPGA designs. This simulator is built on the SLAAC1-V
PCI FPGA board. Radiation-induced upsets are simulated
by artificially changing the contents of the configuration
memory through device partial reconfiguration. The output
of the device is subsequently monitored to determine the
impact of a given configuration memory upset on design
behavior. An important benefit of this SEU simulator is
that it facilitates dynamic circuit testing without the need
of expensive and cumbersome ground-based radiation tests.
While this SEU simulator is convenient, it is essential to
insure that the simulator results match those measured from
tests with radiation sources. Dynamic testing was conducted
at the Crocker Nuclear Laboratory on the SLAAC1-V FPGA
board to validate the accuracy of the SEU simulator. During
accelerator testing, we wanted to obtain approximately one
configuration upset per observation cycle. A proton radiation
source was chosen because protons have a low interaction rate,
making this upset rate feasible.
This paper begins with a discussion of the configuration
bitstream SEU simulator architecture, functionality, and per-
formance. A description of the designs tested is also included,
along with the steps necessary to prepare a design for simulator
or radiation testing. The radiation testing procedure is then
presented, followed by a discussion of results obtained with
the simulator and through accelerator testing. Finally, an
evaluation is made of the accuracy and usefulness of the
configuration SEU simulator.
II. SEU SIMULATOR
A configuration bitstream SEU simulator was created to
test the behavior of FPGA designs in the presence of SEUs
within the configuration memory. Because all information
about an FPGA design is stored in the configuration bitstream,
whenever the state of a bit within the configuration memory
is upset the function of the FPGA design may change. Signals
may be rerouted, logic functions changed, or even the clock
disconnected. The simulator monitors a design to detect if a
configuration upset causes an output error.
The SEU simulator is based on the SLAAC1-V board, a
high-speed FPGA board containing three Xilinx Virtex 1000
FPGAs. The architecture of the SLAAC1-V board is shown
in Figure 2. PE0 is used to provide stimulus to designs in
PE1 and PE2 which operate synchronously and, under normal
circumstances, behave identically. During SEU simulation, the
configuration memory of PE1, the design under test (DUT),
is artificially upset through partial reconfiguration, which
changes the contents of the configurationbitstream. The design
is then executed to determine its true behavior in the presence
of a configuration bitstream SEU. PE0 monitors the circuit
outputs of PE1 and PE2 to determine if the introduction of
an artificial configuration upset into the bitstream of PE1 has
caused an output error. If so, the configuration bit which was
upset is marked as a sensitive location. This entire process is
outlined in Figure 3.
Fig. 2. SLAAC-1V Configuration SEU Simulator architecture.
Fig. 3. Simulator Flow Diagram.
The distribution of the total static cross-section for the
Virtex 1000 part is shown in Figure 4. There are
configuration bits and
tion cross-section of
This yields a total device static cross-section for protons of
?flip flop bits with a satura-
?per bit for protons.
the static sensitivity. This is important because the simulator
tests only the configuration bitstream, it is not able to directly
alter user flip-flop state. Radiation tests cause upsets within
the entire static cross-section of the device, both within the
configuration bitstream and user flip-flops. However, because
the configuration bitstream dominates the static cross-section
(more than 99%) , the SEU simulation results are relevant
for characterizing the dynamic behavior of a design under the
influence of SEUs. It is important in the comparison of the
simulator and the accelerator to understand that the results
are not identical, but approximate due to the small percentage
(0.4%) of the cross-section that is not covered in simulation.
In the process of a typical configuration SEU simulation,
each of these bits is individually upset to determine the effect
?. Clearly the configuration bitstream dominates
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 50, NO. 6, DECEMBER 20033
Fig. 4.Distribution of the static cross-section of the Virtex 1000. There are
?configuration bits tested by the simulator and the
which are untested in simulation.
they have upon the functionality of a given design. Because
there is such a large number of configuration bits within the
bitstream, it is essential that this test be performed as quickly
as possible. The current testing procedure with the SLAAC1-
V board requires 215
meaning that the entire configuration bitstream can be tested
in approximately 20 minutes.
?? to test a single configuration bit,
A. Designs Tested and their Preparation
Two types of designs have been tested with the SEU simula-
tor. The first, a pipelined multiply-and-add design, emphasizes
datapath characteristics, as it is a feed-forward design. The
second, a linear feedback shift register design (LFSR) was
created to study the effects of SEUs on a design with feedback.
The pipelined multiply-and-add design, as illustrated in
Figure 5, consists of eight separate multipliers, whose outputs
are summed to give a final result. The 72-bit wide version of
this design has 36-bit wide
which is 72-bits wide. The problem of having only 36 bits
available for design input (see Figure 2) was resolved by
having input A receive all 36 bits in their original bit ordering,
with input B receiving a permuted version of those same 36
bits. A 36-bit wide version of this same design was created
in order to study the effects of device utilization upon SEU
sensitivity for dynamically tested designs.
? inputs, with an output
O = 8*A*B
Fig. 5.72Mult test design.
The LFSR design consists of clusters of 20-bit wide LFSRs,
with taps at locations given by . Each LFSR cluster contains
DEVICE UTILIZATION FOR THREE TEST DESIGNS. THE LAST ROW SHOWS
THE RESOURCES AVAILABLE IN THE XCV1000 FPGA.
six LFSRs, whose outputs are XORed together to form one
bit of the final circuit output. A 72-bit wide LFSR design was
created, meaning that this design held 72 LFSR clusters, one
for each bit of the output. This design is illustrated in Figure
20 bit LFSR
20 bit LFSR
20 bit LFSR
20 bit LFSR
20 bit LFSR
20 bit LFSR
Fig. 6. 72-bit Linear Feedback Shift Register design.
The device utilization for each of the three test designs
created is given in Table I. The fourth row shows the resources
available in the XCV1000 FPGA. We chose test designs with
different utilization rates and logic types to illustrate that the
dynamic sensitivity of the part is a subset of the static cross-
section. We anticipated that smaller designs would be less
sensitive, and that designs with different architectures, such
as the multiplier and LFSR, would have different sensitivities.
The multiplier test designs are LUT intensive compared to
the LFSR. The 36Mult is approximately 26% the size of the
72Mult test design, but with the same logic architecture.
Proper steps must be taken to prepare a design to be success-
fully tested with the configuration bitstream SEU simulator.
All half-latches must be effectively disconnected from the
design. This is done with the use of a half-latch removal
tool. Also, because readback is used to monitor the state of
the configuration bitstream during radiation testing, no LUT
RAMs may be present in the design. If these two steps are
not taken, inconsistent results may be obtained.
B. Simulator Example
For every design tested, the simulator generates a database
containing the probability that an SEU at a given bitstream
location will cause an output error. The probability of failure
for a bitstream location is computed by dividing the total
number of output errors observed for that location by the
number of artificial upsets that were inserted. The sequence
in which artificial configuration upsets are injected into the
bitstream has no observable impact on the sensitivity of a given
IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 50, NO. 6, DECEMBER 20034
bit. All designs subjected to SEU simulation were operated at
A look at the simulator results conducted on the 72Mult test
design shows that a definite relationship exists between device
utilization and design sensitivity. Figure 7, a circuit schematic
of the 72Mult design, illustrates the device utilization of the
FPGA. For comparison, a map of sensitive locations within
the design can be created by plotting the row and column of
the bit offset of sensitive bits. Figures 8 and 9 are graphical
views of this ‘sensitivity map’ generated from the database
for the 72Mult design. Figure 10 is a probability distribution
function of those bits that were found to be sensitive.
Fig. 7.Routing within the 72Mult test design taken from FPGA Editor.
50010001500 20002500 300035004000 4500
design. A close-up of the boxed region is shown in Figure 9
Map of the sensitive bits obtained with the simulator for the 72Mult
III. RADIATION TESTING
The purpose of this radiation test is to validate the results
obtained from the simulator with the measurements obtained
from the dynamic testing at the accelerator. Dynamic testing
is performed in both the simulator and accelerator to illus-
trate that the functional sensitivity of a design is much less
than what the static sensitive cross-section would suggest. In
addition, the correlation of the accelerator test results with
predictions from the simulator experiment demonstrates the
accuracy of our technique. To perform dynamic testing, our
Fig. 9. A close-up of the map of sensitive bits within the 72Mult test design,
as indicated in Figure 8. The probability of a design failure due to the upset
of a given bit is illustrated by the shade of that bit. The likelihood of failure
appears to be dependent not only upon site utilization, but also upon device
Percentage of total sensitive configuration bits, %
Frequency that a sensitive bit causes an output error, %
Vmult72 sensitive bits = 1.17x106
XCV1000 configuration bits = 5.8x106
72Mult test design (???????
with the SEU simulator; they illustrate that the majority of sensitive bits cause
a design failure 100% of the time.
Histogram of probability of failure of sensitive bits within the
?total sensitive bits). These results are obtained
objective was to operate the designs at speed (2 - 20 MHz) in
the beam with the flux adjusted to slowly introduce upsets. The
functional outputs as well as the configuration bitstream are
continually monitored in the accelerator. When an error in the
configuration bitstream or functional output is identified, the
error is recorded and bitstream errors are repaired. The upset
rate was adjusted to approximately one half the observation
rate to limit the accumulation of upsets between observation
cycles. This more closely represents the environment a design
experiences on-orbit, where we assume that no more than one
upset occurs in the bitstream before it can be repaired. Protons
were chosen as a radiation source because they can provide a
much lower effective rate of upset introduction, due to their
lower interaction rate, than a heavy ion accelerator. Earlier
work demonstrated that the device is sensitive to protons,
so no portion of the circuits was untested due to insufficient
linear energy transfer (LET).