Conference PaperPDF Available

Towards an Integrated GPU Accelerated SoC as a Flight Computer for Small Satellites

Authors:
Towards an Integrated GPU Accelerated SoC as a Flight
Computer for Small Satellites
Caleb Adams
Department of Computer Science
University of Georgia
1510 Cedar St.
Athens, GA 30602
770-314-8422
pieninja@uga.edu
Allen Spain
Department of Electrical
and Computer Engineering
University of Georgia
314 Barnett Shoals Rd.
Athens, GA 30605
770-833-1784
avs81684@uga.edu
Jackson Parker
Department of Electrical Engineering
University of Georgia
1510 Cedar St.
Athens, GA 30602
706-542-2374
jop80923@uga.edu
Matthew Hevert
Department of Mechanical Engineering
University of Georgia
1510 Cedar St.
Athens, GA 30602
303-250-2594
Matthew.Hevert@uga.edu
James Roach
Department of Computer Science
University of Georgia
1510 Cedar St.
Athens, GA 30602
770-845-4787
jhr08134@uga.edu
Dr. David Cotten
Department of Geography
University of Georgia
210 Field St.
Athens, GA 30602
706-542-2374
dcotte1@uga.edu
Abstract—Many small satellites are designed to utilize cutting
edge technology with the goal of rapidly advancing space based
capabilities. As a result, many components take advantage
of developments from the miniaturization of smartphone tech-
nology. Within the past 2 years, the UGA Small Satellite
Research Laboratory has extended this concept into embedded
GPUs for high-performance processing in LEO. Here we show-
case advances in our research of high-performance space-based
computation by integrating a traditional flight computer with
existing miniaturized GPU/SoC systems. Such a system paves
the way for many of NASA’s goals that require space based
AI, neural networks, computer vision, and high performance
computing. Our system fits a standard CubeSat PC/104+ form
factor, and implements many standard protocols such as I2C,
SPI, UART, and RS422. The system also has several GPIO pins,
2 USB-C ports, a micro USB port for flashing, an Ethernet port,
and a micro SD card slot for development. Additionally, the
system is designed to be modular, so that GPU accelerated SoCs
can be stacked to form a distributed system. For our primary
computer, which handles I/O and initializes processes on the
SoC, we choose to use the radiation tolerant Smart Fusion 2 SoC
with an ARM Cortex-M3 processor and a FPGA. In addition
to this primary computer, we use the Nvidia Tegra X2/X2i as
the GPU/SoC workhorse. The primary computer and the TX2i
are designed to share memory space with peripherals mounted
onto the board, so that no significant file transfer is required be-
tween the subsystems. Additionally, Nvidia’s Pascal architecture
enables GPU-CPU or GPU-GPU communication without PCIe,
enabling dense interconnected networks for monitoring and
computation. To address thermal concerns, we cap the TX2i’s
power draw at 7.5 Watts, provide recommendations for thermal
interface materials, and ensure that the primary computer only
enables the GPU/SoC when parallel computation is specifically
requested. Furthermore, radiation mitigation techniques are
explored with ECC, software mitigation techniques, and alu-
minized kapton sheets. In conclusion, this system is a step
towards a miniaturized high-performance flight computer well
suited for future computational demands.
TABLE OF CONTENTS
1. INTRODUCTION................................... 1
2. FORM FACTOR ................................... 2
3. HARDWARE DESIGN .............................. 2
978-1-5386-6854-2/19/$31.00 c
2019 IEEE
4. ARCHITECTURE .................................. 3
5. RADIATION AND THERMAL MITIGATION ....... 3
6. CONCLUSION AND FUTURE WORK .............. 5
APPENDICES......................................... 6
A. CODE AND RESOURCES IN THIS PAPER ......... 6
ACKNOWLEDGMENTS ............................... 6
REFERENCES ........................................ 6
1. INTRODUCTION
Overview
The University of Georgia’s Small Satellite Research Lab-
oratory (SSRL) is utilizing a Nvidia TX2i Graphics Pro-
cessing Unit (GPU) within the Multi-view Onboard Com-
putational Imager (MOCI) Satellite. Although the system
utilizes a GPU, an additional On Board Computer (OBC)
is still required for control and communication with core
avionics. The UGA SSRL has developed a board, the Core
GPU Interface (CORGI), that is capable of interfacing the
Nvidia TX1, Nvidia TX2, or Nvidia TX2i into a PC/104+
compliant CubeSat [1][2][3]. In this paper we discuss the
previously designed CORGI board and elaborate on lessons
learned, including how the CORGI is being used to combine
an OBC and an Nvidia TX2i into one board. We call this new
board, still undergoing development, the Accelerated Flight
Computer (AFC).
The GPU (Nvidia TX2/TX2i) being used is a complete Sys-
tem on a Chip (SoC), capable of running GNU/Linux on an
ARMv8, with a 256 core Pascal GPU. For an exhaustive list
of TX2/TX2i capabilities see the user guide [4] Currently the
TX2/TX2i utilizes CUDA 9.1 [4].
The Microsemi Smart Fusion 2 (SF2) SoC serves as the
central control node for the AFC. The SF2 SoC contains a
166MHz ARM Cortex-M3 Processor with embedded flash
and an FPGA[5]. The SoC has as static power draw of 7mW
and has Single Event Upset (SEU) protected and tolerant
eSRAM and DDR bridges [6]. The SF2’s FPGA has demon-
strated that it operates well in heavy ion and proton radiation
environments, though some mitigation is still required [7].
Additionally, past high-performance space processors have
1
used the SF2 SoC with success [8]. The architecture of the
SF2 SoC makes it ideal for space environments and its wealth
of documentation makes us confident that it will work well
with our design.
Figure 1.A previous iteration of the AFC concept,
known as the CORGI (Core GPU Interface).
Additionally, we seek to adhere to the IEEE Std 1156.4-1997
standard for Spaceborne Computer Modules. This standard
provides requirement levels for thermal performance, pres-
sure, shock, vibration, and radiation [9].
2. FORM FACTOR
The CubeSat form factor is one of the primary considera-
tions in the design of the AFC. The form factor complies
with available deployers such as the Poly Picosatellite Or-
bital deployers (P-POD), and the JEM Small Satellite Or-
bital Deployer (J-SSOD), and is modular to accommodate
the payload and support I/O both spatially and functionally
[10][11]. The CORGI (see Figure 1) board was a design
benchmark which satisfied requirements, such as mission and
development requirements. However the design still required
optimization and additional features to provide a compre-
hensive accelerated heterogeneous computing platform[8].
CORGI’s successor, the AFC, has useful features such as bi-
directional logic shifters to convert the 1.8V (CMOS) logic
from the onboard Nvidia Jetson TX2i [4] to the Smart Fusion
2 3.3v (LVTTL) logic. Shifters are required for serial data
transfer, GPIO flipping, and IC enabling. The AFC will
maintain a PC/104+ form factor (90 x 96 mm)[3] with an
ergonomic cutout to accommodate the high speed LVDS ex-
pansion header (Samtec LSHM-120-04.0-L-DV-A-N-K-TR)
mounted beneath the TX2i module. Tight integration of
both CPU and GPU allows support of all aspects of satellite
performance, such as power, command and data handling,
attitude determination, payload etc. The Smart Fusion 2
ARM Cortex-M3 SoC (M2S150T-1FCG1152) is a low power
and low profile FCBGA (Flip Chip Ball grid Array) com-
bining program accelerators: FPGA, and CPU [8]. The
SF2, primary computer contains an ARM Cortex SoC and is
equipped with a flash based FPGA. The Nvidia TX2/TX2i’s
(50 mm x 87 mm) compact design provides high density
compute, and parallelized Pascal microarchitecture. A 400-
pin compatible board-to-board connector will interface with
the TX2i module via a 8x50 connector (SEAM-50-02.0-S-08-
2-A-K-TR). The Jetson TX2i was added to the AFC due to its
form factor and recent usage in high performance computing
(HPC) in industrial environments, long life, and ECC mem-
ory [4]. The AFC form factor maximizes compute, while
eliminating general purpose parts sold with COTS On Board
Computers (OBC). This has the added benefit of simplicity of
development, and reduced quiescent current draw.
Table 1.Processors
Smart Fusion2:
ARM Cortex-M3 SoC (M2S150T-1FCG1152)
NVIDIA Jetson TX2:
-2x Denver ARM Cortex-A57
-256-core Pascal GPU
Table 2.Memory
SF2: 4x256 DDR3 Memory Bank
TX2: 8GB LPDDR4 and 32 GB ECC support
Both:
-2 Gb (Cypress CYRS16B256)
-1 Gb SPI Flash on SPI0 via FPGA MSS
-1Gb Flash on SPI1 via FPGA fabric
Table 3.IO Interfaces
PC/104+, 12C, SPI, GPIO
Expansion header (QSPI)
2x RJ-45
2x USB 3.0 Type C
Micro USB (FTDI)
3. HARDWARE DESIGN
Hardware
As shown in Figure 3, H1 and H2 are the main PC/104+
CubeSat headers. These route peripheral components onto
the bus, as well as routing the PC/104+. The H1 and H2
headers on the AFC also act as pass-throughs, so there is no
break in communication or power. The Smart Fusion 2 shares
all primary communication lines and acts as the control node
for the command and data handling (CDH) subsystem of the
satellite. In addition to two RJ45 Ethernet receptacles the
AFC board also makes a QSPI expansion header available for
added support and 2x USB Type C for debugging the TX2i.
Prototyping
The SmartFusion 2 advanced development kit [12] was used
for the prototyping of the integrated CPU-GPU board. The
SmartFusion 2 advanced development kit contains a wide
range of headers (SPI, I2C, GPIO, UART, etc.) [12]. The
versatility of this development kit makes it ideal for prototyp-
ing.
The Nvidia TX1/TX2 development kit includes a developer
board that is compatible with the Nvidia TX2i [4]. The
developer board, connected to the Nvidia TX2/TX2i, can be
interfaced with the Smart Fusion 2 advanced development
kit via 10/100/1000 PHY Ethernet connection, SPI, LVTTL
UART, I2C, PCIe, or GPIO[4][12].
2
Figure 2.AFC 3D design back
Figure 3.AFC 3D design front
4. ARCHITECTURE
Hardware Architecture
With ubiquitous computing and 10/100/1000 Ethernet
Switching, Ethernet provides wired communication for the
satellite bulk data transfer between the SF2 and the one
or more TX2i. This design contains carrier level Ethernet
whose data link and physical layer protocol are described
by the IEEE 802.3. [13] This design utilizes a node switch
which acts as a physical medium.The SF2, being the pri-
mary distributer of the workload, design contains SEU Pro-
tected/Tolerant Memories i.e: eSRAMs, DDR3. Its robust
nature played a considerable roll in its selection as the pri-
mary processor on the accelerated flight computer. In the
event of one or all of the TX2is destruction, the SF2 will be
able to perform I/O monitoring and control duties without a
co-compute device.
This option, although functional, will diminish payload ca-
pabilities. In response, the system-on-chip (SoC) field pro-
grammable gate array (FPGA) microcontroller subsystem
(MSS) includes a watchdog which provides a system which
needs no operating system to detect and respond to lockups
caused by heavy ion bombardment [12]. The SF2 will
utilize 4 x 256 DDR3 (MT41K256M8DA-125 IT:K) external
memory banks providing volatile memory. In addition, the
TX2i modules and SF2 share RAD NOR SPI flash memory
(Cypress CYRS16B256). The SF2 will allow mediated
read/write access with the purpose of sharing mutually rele-
vant files. This functionality is defined in hardware via MUX
whose selection is determine by GPIO pins from SF2 SoC,
active high.
Software Architecture
This section details how software will make use of the hard-
ware system presented above in order to provide a modular
fault tolerant accelerated computing platform. The Smart
Fusion 2 will act as the main control node for the whole
system and will make use of four main interfaces to interact
with the TX2is. These interfaces will be a Gb Ethernet
local network, GPIO, SPI, and shared flash memory that the
Smart Fusion 2 and TX2is will access via multiplexed SPI
connections. The ethernet line will be used by the Smart
Fusion 2 to distribute computation tasks and data to the
TX2i’s. These ethernet communications will take the form
of multicast messages, as each TX2i will need to receive and
direct messages for communication to individual TX2is. The
computation tasks to be distributed to each TX2i will take the
form of CUDA code. The TX2is will make use of the ethernet
line or GPIO to kick the watchdogs on the Smart Fusion 2
FPGA and to log their progress or telemetry. The Smart
Fusion 2 will use the SPI accessed shared flash memory as
its primary persistent memory where it will store tasks and
data to distribute to the TX2is. The TX2is will make use of
the shared flash memory by writing computational results to
it.
5. RADIATION AND THERMAL MITIGATION
System Hardware Mitigation and Shielding
For radiation and thermal mitigation in Low Earth Orbit
(LEO) we recommend utilizing the Dunmore Aerospace
“Satkit” which contains the standard STARcrest MLI mate-
rials cut into manageable sizes for small satellite developers.
This allows one to develop their own thermal protection
blankets according to various mission requirements. The kit
includes an outer layer material, inner layer material, first
surface tape, and clear polyimide tape[14]. This kit is optimal
for small satellite systems as it is small, cheap, and easily
customizable. More specifications of the materials are listed
below (See Figure 4) and values were calculated from the
Solar Radiation, Earth’s IR radiation, and Albedo Radiation
equations [15].
q=Gsscos (1)
q=T4
eIRF e (2)
q=Gs(AF )sFecos (3)
3
Figure 4.The Hardware Design of the Cubesat GPU/SoC-CPU system
The heat flux due to the LEO sources using equations 1, 2,
and 3 was calculated with the help of data from literature [15].
Values have an expected error of ±0.4. Values for the solar
radiation were determined using the mean values provided by
data from the World Radiation Center in Davos Switzerland
[16].
The necessary thickness of the radiation shield was calculated
in part using values provided by Dunmore. The primary
source of heat flux is solar, these values vary on an annual
basis due to the Sun’s elliptical orbit. Meaning that the
maximum and minimum amount of flux expected from the
Sun would range between 1322 and 1414w/m2. The target
ambient temperature inside the CubeSat is 293.15 K (20
degrees C). This is calculated with the following formula:
Q=TKA
L(4)
Where Qis the heat flux into the system, Kis the thermal
conductivity of the material, A is the area in meters squared,
and Lis the thickness of the radiation shielding. Assum-
ing a thermal conductivity of 0.014 and a Tof 101K.
The expected required thickness of the radiation shielding is
1.03438 ·107m2. Radiation from Free Molecular Heating
(FMH) was determined to be negligible for the stage in which
the CubeSat would be launched from the ISS. FMH is almost
exclusively encountered during launch ascent just after the
booster’s payload fairing is ejected.
Radiation Mitigation
The radiation shielding thickness (See Equation 4) is also
driven by the 1997 IEEE Standard for Environmental Specifi-
cations for Spaceborne Computer Modules [9]. We design for
level I radiation in preperation for the LEO environment. An
additional benefit of the Dunmore Aerospace “Satkit” is that
it meets this standard while accruing minimal mass gains.
Proper shielding does not have to incur heavy mass gains,
Table 4.Recommended Dunmore Aerospace Satkit Part
VDA / 200 GA Polyimide
Tensile Strength 24000 psi
Elongation 50%
Thickness 50.8 micron
Density 1.42 g/cc
Yield 13.8 m·m/kg
Weight/Area 72 g/m2
Operating Temp -250 - 400 C
Metalization 99.99 % pure aluminium
VDA / 200 GA Polyimide
Tensile Strength 26000 psi
Elongation 110 %
Thickness 6.35 micron
Density 1.39 g/cc
Yield 124.9 m·m/kg
Weight/Area 8 g/m2
Operating Temp -250 - 150 C
Metalization 99.99 % pure aluminium
VDA / 25 GA PET / VDA, Embossed & Perforated
Thickness 76.2 microns
Yield 10.4 m·m/kg
Weight/Area 96 g/m2
Operating Temp -40 - 220 C
Metalization 99.99 % pure aluminium
100 GA Polyimide / 966 PSA
Thickness 76.2 microns
Yield 10.4 m·m/kg
Weight/Area 96 g/m2
Operating Temp -40 - 220 C
Metalization 99.99 % pure aluminium
and in fact shielding that is too thick can increase the effects
of some kinds of radiation events. This is due to the higher
levels of secondary particles created when a high-energy
GCR particle impacts a thick shield [17]. If necessary, a
4
properly designed shield may act as a heat sink due without
exceeding mass limitations. This reduction of a device’s
operating temperature can greatly reduce the risks posed to
that device by radiation [17].
When working with Commercial Off The Shelf (COTS) elec-
tronic components for use in the space environment, some
of the problems that must be addressed are data integrity
performance and accuracy in high radiation environments. In
our design, the TX2i, one of the Tegra SoCs, is one of the
more vulnerable parts with respect to this, due to its highly
dense hardware design[18]. As stated above, much can be
done at a hardware level but often times additional software
mitigation is needed.
One of the major problems radiation presents for software is
its effect on memory. Overtime as bit flips [15] [9] aggregate,
they will corrupt information and make some systems unus-
able. To address these issues, in the AFC we plan on modify-
ing the file system and bootloader. We will triplicate the file
system on the TX2i so that any part of it which is corrupted
will be able to use the principle of triple modular redundancy
(TMR) to automatically correct damage. In addition to this,
we will use the U-Boot bootloader software to modify the
bootloading process with TMR [19]. The bootloader will
store 3 copies of the operating system (OS) image for the
TX2i and a hash for each image. At boot time the bootloader
will recalculate the hash for each OS image it attempts to
load and compare the calculated hash against the stored hash
to determine if the OS has been corrupted. If corruption is
detected then boot loader will attempt to load the next OS
image. If all OS images are determined to be corrupted, the
bootloader shall attempt to construct an uncorrupted image
by bit voting between the corrupted images.
Our design uses the SF2 FPGA as a trusted control node
due test results showing high levels of radiation tolerance [7].
Thus, we plan to have the Smart Fusion 2 FPGA operate as a
watchdog for each TX2/TX2i. If the Smart Fusion 2 detects
that one of the TX2/TX2i’s has anomalous power draw levels
(over 7.5 Watts), it will send a command over ethernet (via
the SF2 SoC) commanding the TX2i to reduce GPU usage
until the power level stabilizes.
Thermal Mitigation
Simulations have shown, given a large enough mounting
structure, that the Nvidia TX2 is capable of dissipating heat
effectively. To achieve this goal it is imperative that the TX2’s
Thermal Transfer Plate (TTP) is adequately interfaced into
the heat dissipating mass[1]. To interface the TTP with the
heat dissipating mass we highly recommend the use of a low
thickness and high conductance thermal interface material
(TIM) that adheres to the NASA standards for collected
volatile condensible materials (0.1% CVCM) and total
mass loss (1% TML) [20]. Previous findings have sug-
gested that the Carbice Space TIM is ideal for these purposes
due to its low thickness (0.065mm) and high conductance
(13,330Wm
2K1) [1].
It is important to note that all thermal simulations are run
with a heat load according to subsystem power draws and
assume 0% efficiency for a worst case scenario. The steady
state thermal simulation clearly shows that the GPU’s core
temperature has been lowered with the addition of the TIM.
The overall maximum has been brought down from around
160C, to under 50C. While this model did not include
the entire spacecraft structure as a conduction medium, this
would only serve to decrease the temperature further, as
Figure 5.The Nvidia TX2 daughter-board with the
Parker Series SoC Exposed. No TIM is used in this
simulation.
Figure 6.The Nvidia TX2 daughter-board with the
Parker Series SoC Exposed. TIM is used in this
simulation
conduction is a much faster heat transfer mechanism than
radiation.
Figure 7.The Nvidia TX2 with TTP integrated and TIM
used.
However, some caution must be used with these results. The
thermal environment of the satellite is inherently transient, so
the ambient temperature assigned to the model here is likely
inaccurate. This also means a “steady state” analysis might
not necessarily be appropriate. However, the purpose of
this analysis is not to provide conclusive thermal information
about the AFC, rather to serve as a data point for design,
and to show that under extremely approximate conditions, the
AFC will be able to operate in orbit.
6. CONCLUSION AND FUTURE WORK
The UGA SSRL’s CORGI design, the starting point for de-
signing the AFC, has made the usage of the Nvidia TX2/TX2i
more feasible in LEO. The recommendations provided in the
5
Figure 8.The Nvidia TX2 with TTP integrated, no TIM
is used.
paper regarding radiation and thermal mitigation provide a
starting point for those who wish to utilize the Nvidia TX1,
TX2, or TX2i in LEO. Additionally, when designing a fully
integrated system with the Nvidia TX2/TX2i, the use of a
SmartFusion2 SoC FPGA is highly recommended. It is well
understood, well documented, radiation tolerant, and operates
on low power. The combination of the SmartFusion2 with the
Nvidia TX2/TX2i has the potential to meet many of NASA’s
goals that require space based AI, neural networks, computer
vision, and high performance computing[21].
Future work will entail testing of the aforementioned compo-
nents. The system shall be able to halt and restart actions
on the subsystem and utilize Paralleled Thread Execution
assembly to checkpoint GPU operations. Additional methods
for thermal dissipation will be simulated and those mentioned
above will be tested in our thermal vacuum chamber. Radia-
tion testing (heavy ion and proton) will be performed during
heavy computational load. The system will be used to form
a modular computational cluster and distributed computation
will be further tested in our vacuum chamber while the system
is under heavy load.
APPENDICES
A. CODE AND RESOURCES IN THIS PAPER
The resources used in this paper can be requested at any point
by contacting the authors of this paper.
ACKNOWLEDGMENTS
The authors would like to thank the Georgia Space Grant
Consortium for funding these GPU research projects and
the Air Force Research Laboratory’s University Nanosat
Program for giving us tremendous opportunities and for
funding the projects that led us to this point. Additionally
the authors would like to thank Erick Gavilanes for making
the initial design for the CORGI in 2017 and getting us to
a point where we could perform iterations and follow up
research. Thanks to Casper Versteeg for providing significant
knowledge and results relating thermal mitigation. Nir Patel,
Nicholas Heavner, and Michael Buzzy did great work by
contributing to the thermal models. A big thanks Justin
Heimerl, for supplying CAD and working on electrical design
and to Sydney Whilden and Adam King for doing initial
radiation research and laying the foundations for this paper
in that respect. A special thanks to Dr. Deepak Mishra for
helping support the University of Georgia’s Small Satellite
Research Laboratory from the beginning. Another thank you
to Roger Hunter.
REFERENCES
[1] C. Versteeg, “Thermal management and design of high
heat small satellite payloads,” 32nd Annual AIAA/USU
Conference on Small Satellites, 2018.
[2] C. Adams, A near real time space based computer vi-
sion system for accurate terrain mapping,” 32nd Annual
AIAA/USU Conference on Small Satellites, 2018.
[3] (2003) Pc/104-plus specification. [Online]. Avail-
able: www.winsystems.com/wp-content/uploads/specs/
PC104PlusSpec.pdf
[4] (2017) Jetson tx2 developer kit user guide.
[Online]. Available: www.developer.nvidia.com/
embedded/jetpack
[5] (2015) Arm cortex-m3 processor tech-
nical reference manual. [Online]. Avail-
able: www.infocenter.arm.com/help/topic/com.arm.
doc.ddi0337e/DDI0337E cortex m3 r1p1 trm.pdf
[6] (2018) Product brief smartfusion2 soc fpga. [Online].
Available: www.microsemi.com/document-portal/
doc view/134215-ug0557-smartfusion2- soc-fpga-
advanced-development-kit-user-guide
[7] (2014) Igloo2 and smartfusion2 65nm commercial
flash fpgas interim summary of radiation test results.
[Online]. Available: www.microsemi.com/document-
portal/doc view/134103-igloo2-and- smartfusion2-
fpgas-interim-radiation-report
[8] F. Bruhn, “Introducing radiation tolerant heterogeneous
computers for small satellites,” 32nd Annual AIAA/USU
Conference on Small Satellites, 2015.
[9] “Ieee standard for environmental specifications for
spaceborne computer modules,” IEEE Std 1156.4-1997,
1997. [Online]. Available: www.ieeexplore.ieee.org/
stamp/stamp.jsp?arnumber=603627&tag=1
[10] (2016) 6u cubesat design specification. [Online].
Available: http://org.ntnu.no/studsat/docs/proposal 1/
A8%20-%20Cubesat%20Design%20Specification.pdf
[11] “Nanoracks cubesat deployer (nrcsd) interface
definition document (idd). nr-nrcsd-s0003, 2018.
[Online]. Available: www.nanoracks.com
[12] (2016) Smartfusion2 soc fpga advanced development
kit user guide. [Online]. Available: www.microsemi.
com/document-portal/
[13] (2008) Ethernet theory of operation. [Online]. Avail-
able: www.microchip.com/downloads/en/appnotes/
01120a.pdf
[14] Dunmore aerospace satkit. [Online]. Available: www.
dunmore.com/products/satkit-mli.html
[15] G. D. Badhwar, “The radiation environment in low-
earth orbit,” Radiation Research, vol. 148, no. 5,
pp. S3–S10, 1997. [Online]. Available: www.jstor.org/
stable/3579710
[16] “World radiation data centre, 2018. [Online].
Available: www.re3data.org/repository/r3d100010177
[17] R. H. Maurer, M. E. Fraeman, M. N. Martin, and
D. R. Roth, “Harsh environments: Space radiation en-
6
vironment, effects, and mitigation,Johns Hopkins APL
Technical Digest, vol. 28, no. 1, pp. 17–29, 2008.
[18] E. Wyrwas, “Body of knowledge for graphics
processing units (gpus) nepp-bok-2018,” 2018.
[Online]. Available: www.nasa.gov/sites/default/files/
atoms/files/2017-8-1 stip final-508ed.pdf
[19] “The denx u-boot and linux guide (dulg) for
canyonlands,” 2018. [Online]. Available: www.denx.
de/wiki/DULG/Manual
[20] R. R. J. Roe, “Standard materials and processes require-
ments for spacecraft,” Technical Report NASA-STD-
6016A, November 2016.
[21] (2017) Nasa strategic technology investment plan.
[Online]. Available: www.nasa.gov/sites/default/files/
atoms/files/2017-8-1 stip final-508ed.pdf
Caleb Adams Caleb Adams is a Grad-
uate student in Computer Science and
co-founder of the UGA Small Satel-
lite Research Laboratory, which now
has 2 satellite missions. Caleb’s re-
search mostly focuses on high perfor-
mance computation systems in space en-
vironments for AI and Computer Vision.
He has also worked on audio telemetry
systems for Orion at NASA, was also a
google glass beta tester, and a TEDxUGA speaker.
Allen Spain Allen Spain is currently
a M.S. Student in the School of Elec-
trical and Computer Engineering at the
University of Georgia. He specializes
in electronic circuit design, and is the
primary designer of Electronics Inter-
face Boards for the UGA Small Satellite
Laboratory. He is currently a Research
Assistant at the University of Georgia
focusing on Biomimetic Photonic algo-
rithms.
Jackson Parker Jackson Parker re-
ceived his B.S. in biological engineering
in spring of 2018 from the University of
Georgia, where he is currently a Mas-
ters student in Electrical and Computer
Engineering. Current research activities
and interests include accelerated com-
putation, machine learning, algorithmic
development, and computer vision. He
plans on furthering his knowledge in
those areas of interest while working on his M.S. and then
pursue a PhD in computer science.
Matthew Hevert Matthew Hevert is
currently pursuing a Bachelors of Sci-
ence in Mechanical Engineering at the
University of Georgia with an antici-
pated graduation in 2020. He has been
working at the Small Satellite Research
Laboratory performing work in optics,
thermodynamics, and mechanical radia-
tion mitigation. He plans on continuing
his education after graduation in optical
sciences and engineering where he one days hopes to help
design the next generation of space-based telescopes.
James Roach James Roach is an un-
dergraduate computer science student
graduating from the University of Geor-
gia in May 2019. He has been working
at the small satellite research laboratory
since fall 2015. He has also worked on
small satellites for Space Dynamics Re-
search Laboratory. Current research in-
terests include accelerated computation,
the effect of radiation on electronics, and
fault tolerance in cubesat platforms.
David Cotten David L. Cotten, Ph.D.,
received a B.S in Physics from Louisiana
State University in 2005. He currently
serves as the manager and Co-PI of the
UGA’s Small Satellite Research Labora-
tory and is an Assistant Research Sci-
entist at the Center for Geospatial Re-
search in the Geography Department at
UGA. He graduated from UGA in 2011
with his doctorate in Physics and As-
tronomy. His research as a Post-Doctoral Associate focused
on surface-atmosphere exchange, and he is currently using
remote sensing (multispectral/hyper spectral sensors) and
micrometeorology techniques to quantify carbon storage in
wetland regions at both the local and regional scales. Dr.
Cotten is also using unmanned aerial vehicles, air photos,
and satellite imagery to build 3D models of terrestrial objects
using photogrammetric Structure from Motion.
7
... The use of COTS components in Low Earth Orbit (LEO) missions and CubeSats relies on the partial shielding provided by the Earth's magnetosphere and/or the short mission lifetime, which limit the damage or unavailability of electronics due to radiation. In this context, FPGAs [13,14,15,16], GPUs [17,18,19,20,21] and VPUs [22,23,24,25] are evaluated as accelerators, while COTS devices, e.g., Intel's Myriad 2 VPU, are subjected to radiation tests [26]. A second challenge for the space industry is the wider adoption of AI, which is currently limited to offline/ground data processing instead of on-board processing, mostly due to insufficient computational power and qualification issues when deployed in orbit [26]. ...
... Moreover, the literature includes FPGA-GPU coprocessing architectures. The hybrid FPGA-GPU architecture of [19] employs Nvidia's Tegra X2/X2i GPU as main accelerator. The heterogeneous architecture of [20] integrates a SoC FPGA for SpaceWire I/O transcoding, the AMD SoC (CPU & GPU) for acceleration, and optionally, a VPU for AI deployment. ...
Preprint
The challenging deployment of Artificial Intelligence (AI) and Computer Vision (CV) algorithms at the edge pushes the community of embedded computing to examine heterogeneous System-on-Chips (SoCs). Such novel computing platforms provide increased diversity in interfaces, processors and storage, however, the efficient partitioning and mapping of AI/CV workloads still remains an open issue. In this context, the current paper develops a hybrid AI/CV system on Intel's Movidius Myriad X, which is an heterogeneous Vision Processing Unit (VPU), for initializing and tracking the satellite's pose in space missions. The space industry is among the communities examining alternative computing platforms to comply with the tight constraints of on-board data processing, while it is also striving to adopt functionalities from the AI domain. At algorithmic level, we rely on the ResNet-50-based UrsoNet network along with a custom classical CV pipeline. For efficient acceleration, we exploit the SoC's neural compute engine and 16 vector processors by combining multiple parallelization and low-level optimization techniques. The proposed single-chip, robust-estimation, and real-time solution delivers a throughput of up to 5 FPS for 1-MegaPixel RGB images within a limited power envelope of 2W.
... Low Earth Orbits (LEOs) are evolving into a new class of "mobile computing clusters". The recent technological advances in rocket reusability and affordable nanosatellites have sharply decreased the cost of launching computing payloads to space, stimulating a surge in operational satellite constellations equipped with cameras, sensors, and GPUs [1][2][3]. These in-orbit edge computing resources empower satellites to locally process runtime raw data (e.g., 20TB/day of Earth observation images, sensor data, and system logs) rather than transmitting them to the ground stations for remote processing, thus saving precious satellite downlink bandwidth [4][5][6][7][8] and facilitating real-time satellite tasks like disaster detection [9], climate monitoring [10], and precision agriculture [11]. ...
... Then, the onboard model's runtime inference(Figure 4) depends on DNN-specific data flow, which is determined by two parts.(1)The computational graph (G) determines how typical DNN layers are arranged in memory like convolution (Conv) layers, batch norm (BN) layers, activation (e.g., ReLU) layers, etc.(2) ...
Preprint
We are witnessing a surge in the use of commercial off-the-shelf (COTS) hardware for cost-effective in-orbit computing, such as deep neural network (DNN) based on-satellite sensor data processing, Earth object detection, and task decision.However, once exposed to harsh space environments, COTS hardware is vulnerable to cosmic radiation and suffers from exhaustive single-event upsets (SEUs) and multi-unit upsets (MCUs), both threatening the functionality and correctness of in-orbit computing.Existing hardware and system software protections against radiation are expensive for resource-constrained COTS nanosatellites and overwhelming for upper-layer applications due to their requirement for heavy resource redundancy and frequent reboots. Instead, we make a case for cost-effective space radiation tolerance using application domain knowledge. Our solution for the on-satellite DNN tasks, \name, exploits the uneven SEU/MCU sensitivity across DNN layers and MCUs' spatial correlation for lightweight radiation-tolerant in-orbit AI computing. Our extensive experiments using Chaohu-1 SAR satellite payloads and a hardware-in-the-loop, real data-driven space radiation emulator validate that RedNet can suppress the influence of radiation errors to \approx 0 and accelerate the on-satellite DNN inference speed by 8.4%-33.0% at negligible extra costs.
... First, learning-based techniques offer an appealing paradigm for solving problems characterized by stochastic spacecraft dynamics, multi-step decision-making, and potentially highly non-convex objectives [5], [6]. Second, the computational overhead of using trained ML models during inference is low, potentially compatible with the limited computational capabilities available onboard spacecrafts [6], [7]. However, learning-based methods are often sensitive to distribution shifts in unpredictable ways, whereas optimization-based approaches are more readily characterized in terms of robustness and out-of-distribution behavior. ...
Preprint
Effective trajectory generation is essential for reliable on-board spacecraft autonomy. Among other approaches, learning-based warm-starting represents an appealing paradigm for solving the trajectory generation problem, effectively combining the benefits of optimization- and data-driven methods. Current approaches for learning-based trajectory generation often focus on fixed, single-scenario environments, where key scene characteristics, such as obstacle positions or final-time requirements, remain constant across problem instances. However, practical trajectory generation requires the scenario to be frequently reconfigured, making the single-scenario approach a potentially impractical solution. To address this challenge, we present a novel trajectory generation framework that generalizes across diverse problem configurations, by leveraging high-capacity transformer neural networks capable of learning from multimodal data sources. Specifically, our approach integrates transformer-based neural network models into the trajectory optimization process, encoding both scene-level information (e.g., obstacle locations, initial and goal states) and trajectory-level constraints (e.g., time bounds, fuel consumption targets) via multimodal representations. The transformer network then generates near-optimal initial guesses for non-convex optimization problems, significantly enhancing convergence speed and performance. The framework is validated through extensive simulations and real-world experiments on a free-flyer platform, achieving up to 30% cost improvement and 80% reduction in infeasible cases with respect to traditional approaches, and demonstrating robust generalization across diverse scenario variations.
... Different CPU+GPU SoCs, such as the Nvidia Jetson X2i [124] and Xavier NX [125] series and the AMD Embedded G-Series SoC family [126], have been proposed in many CubeSat payload processing systems. The educational nanosatellite Multiview Onboard Computational Imager (MOCI), developed by University of Georgia and planned to be launched by 2024, is one of the first CubeSat to include a Nvidia X2i in the design of its Accelerated Flight Computer (AFC; [127,128]). Similarly, the Space Edge One (SE-1), a small form factor (0.25 U) on-board computing system developed by Spiral Blue (Australia; [129]), utilizes a Jetson Xavier NX and underwent space testing in 2023. ...
Article
Full-text available
Over the past three decades, the acceptance of higher risk thresholds within the space industry has facilitated the widespread integration of commercial off-the-shelf (COTS) components into avionics and payloads, leading to a remarkable transformation in the design of space missions. This transformation has led to the emergence of the New Space Economy and the widespread adoption of lean or small satellites in general, particularly CubeSats. CubeSats are now widely used in commercial, scientific, and research applications due to their versatility, affordability, simplicity of development, and accelerated development timelines. On-board computing plays a crucial role in the design of CubeSat missions, as increasingly high-performance computational requirements are needed to meet the challenges of future missions. This paper systematically reviews the state-of-the-art of CubeSat Command and Data Handling (C&DH) sub-system, covering both hardware components and flight software (FSW) development frameworks. It presents an analysis of the key features and recent developments of on-board computers (OBCs) in commercial and academic institutional projects funded by governments, agencies and public institutions. It further examines the effects of space radiation on avionics components and discusses the main fault-tolerance techniques used in CubeSat platforms. Finally, this paper highlights trends and hazards for future CubeSat avionics and identify potential directions for future developments in high-performance on-board computing. By synthesizing contemporary research and industry insights, this paper aims to shed light on CubeSat OBC design, providing an overview of the existing technology landscape and the challenges to be addressed for next-generation mission needs.
... In the avionics and space industry, GPUs are utilized as underlying engines to support vision-based navigation and mid-air object detection [138]. Additionally, GPUs aid in computationally intensive tasks, including flight management and data processing, playing a vital role in the industry [139,140]. Furthermore, there are plans to employ GPUs in railway systems as a powerhouse to enable trains' safe and dynamic management based on environmental and geometrical parameters [141]. Lastly, GPUs are utilized in various industrial applications, including industrial control robots and predictive equipment maintenance [142]. ...
Thesis
Full-text available
In the rapidly evolving landscape of nanotechnology, where innovations promise groundbreaking advancements in various industries, ensuring the reliability and safety of digital circuits becomes paramount. While applicable to the broad category of commercial off-the-shelf products, this becomes significantly more evident when examining domains or industry sectors, such as automotive, aviation, railways, and the biomedical sector, that fall into the category of safety-critical applications. In these cases, the probability that a fault may activate an error and propagate to a failure that would endanger human lives or cause environmental damage should be carefully evaluated and kept under predefined thresholds. To achieve this result, the manufacturers must comply with strict safety standards and procedures that mandate rigorous coverage thresholds and comprehensive testing protocols. From end-of-manufacturing up until the in-field phase, each integrated circuit (IC) is subjected to several testing procedures to ensure that it meets stringent quality standards, functions reliably within specified parameters, and remains resilient to various environmental conditions throughout its operational lifespan. Design-for-testability (DfT) techniques are incorporated during the design phases of electronic circuits to enhance the testing process. However, despite the presence of powerful electronic design automation (EDA) utilities, such as automatic test pattern generation (ATPG) tools, intended for use alongside DfT-compliant designs during testing, the relentless evolution of technology brings about faster, smaller, and denser circuits. This evolution renders certain utilities inadequate as the complexity of the test procedure significantly increases, as seen with Burn-In (BI) test. Burn-In, an omnipresent step in the test chain for products intended for use in safety-critical domains, was, until recently, conducted in its traditional static format. Notwithstanding its effectiveness, static BI test became less effective for newer, denser technologies as in its static form it was found not to fully exercise all internal parts of the ICs. Hence, it evolved into new dynamic forms, where stress stimuli are applied in an internal manner on top of the external temperature and voltage increase. However, the generation of appropriate stress-inducing stimuli is a costly and arduous task for the test engineers, due to the lack of automation to aid the generation process. Another test domain that could substantially benefit from automation is the in-field test. Continuous in-field testing enables the detection of faults or anomalies that may occur over time. Early detection of potential issues allows for proactive maintenance or corrective measures, reducing the risk of system failures in critical situations. However, the task of developing appropriate software test libraries (STLs) for such scenarios is typically a task that requires a lot of manual effort from the perspective of the test engineer. In fact, not only the test must consider parameters such as application time and memory footprint but it must also avoid targeting untestable faults of the design. This means that the test should only focus on those faults that are able to produce a failure in the operating scenario, ignoring those that can not produce any (critical) failure. This PhD thesis proposes solutions, based on Formal Methods (FMs), addressing the aforementioned test topics. The manuscript is organized in three main parts. The initial part provides an introduction and overview of the imperative need for testing and reliability in the modern digital era. It delves into the distinct testing areas that form the focus of this thesis. The second part includes the three main contributions of the thesis. A section proposing FM-based solutions for dynamic BI test stress stimuli generation is first presented. These methods consider various switching activity metrics, and their effectiveness is showcased by applying them on scalar pipelined processors. The second contribution regards FM-based solutions targeting the identification of functionally untestable faults under the stuck-at and the cell-aware fault models. Lastly, the final contribution regards methodologies that aid the generation of STLs for microprocessors and GPUs. The last part provides the conclusions of the overall work.
... The NVIDIA Jetson TX2 is a state-of-the-art AI on-the-edge device [25], equipped with multiple GPU and CPU cores that enable it to handle heavy loads, making it an efficient implementation of Convolutional Neural Networks (CNNs). Due to its impressive capabilities, it has already been considered for use in space applications [26]. The objective of this study is to explore the potential of utilizing PRISMA HS images for real-time fire detection and to facilitate effective crisis management by introducing on-board calculations for early warnings. ...
Conference Paper
Full-text available
This research explores the potential use of artificial intelligence techniques and edge computing approaches to detect wildfires directly from satellite platforms. The study is based on PRISMA (Hyperspectral Precursor of the Application Mission), an Italian hyperspectral satellite launched in 2019 that provides hyperspectral imagery in the spectral range of 0.4-2.5 µm with an average spectral resolution of less than 10 nm. The paper presents new results related to the Australian fires that occurred in December 2019 in New South Wales, acquired by PRISMA on December 27, 2019. The paper aims to investigate the practicality of deploying a one and three-dimensional convolutional neural network (CNN) models, as previously proposed by previous authors' works, with the assistance of an Nvidia Jetson TX2 as a testing hardware accelerator. This experiment explores the potential of utilizing on-the-edge deployment for this technology. This study aligns with efforts to improve the computational capabilities and autonomy of satellites, which could pave the way for future satellites or constellations with a specific focus on remote sensing and the provision of timely and reliable alerts.
... On the other hand, DNNs are increasingly employed in safety-critical disciplines like healthcare [1], automotive [2], avionics [3], space [4], railway [31], and industry [6] to extract useful information from complex raw data. Hence, the dependability of deployed accelerator-based systems is rising in importance. ...
Article
Full-text available
Nowadays, artificial intelligence (AI) and deep learning (DL) progressively adapt to various spheres of our lives. These disciplines contain safety-critical applications such as autonomous driving with a high risk of human injury in the case of malfunction, requiring a high promise of dependability. Even the dependability becomes more crucial as shrinking CMOS technology feature size worsens the resilience concerns due to factors like aging. This paper addresses the overarching dependability issue of advanced deep neural networks (DNN) accelerators from the aging perspective. Especially, a comprehensive survey and taxonomy of techniques used to evaluate and mitigate aging effects are introduced. We cover different aging effects like permanent faults, timing errors, and lifetime issues. We review research by the layer-wise approach and categorize several resilience classes to bring out major features. The concluding part of this review highlights the questions answered and several future research directions. This study is expected to benefit researchers in different areas of DNN deployment, especially the dependability of this emergent paradigm.
Article
The challenging deployment of Artificial Intelligence (AI) and Computer Vision (CV) algorithms at the edge pushes the community of embedded computing to examine heterogeneous System-on-Chips (SoCs). Such novel computing platforms provide increased diversity in interfaces, processors and storage, however, the efficient partitioning and mapping of AI/CV workloads still remains an open issue. In this context, the current paper develops a hybrid AI/CV system on Intel’s Movidius Myriad X, which is an heterogeneous Vision Processing Unit (VPU), for initializing and tracking the satellite’s pose in space missions. The space industry is among the communities examining alternative computing platforms to comply with the tight constraints of on-board data processing, while it is also striving to adopt functionalities from the AI domain. At algorithmic level, we rely on the ResNet-50-based UrsoNet network along with a custom classical CV pipeline. For efficient acceleration, we exploit the SoC’s neural compute engine and 16 vector processors by combining multiple parallelization and low-level optimization techniques. The proposed single-chip, robust-estimation, and real-time solution delivers a throughput of up to 5 FPS for 1-MegaPixel RGB images within a limited power envelope of 2 W.
Conference Paper
Full-text available
The Multiview Onboard Computational Imager (MOCI) is a 3U cube satellite designed to convert high resolution imagery, 4K images at 8m Ground Sample Distance (GSD), into useful end data products in near real time. The primary data products that MOCI seeks to provide are a 3D terrain models of the surface of Earth that can be directly compared to the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) v3 global Digital Elevation Model (DEM). MOCI utilizes a Nvidia TX2 Graphic Processing Unit (GPU)/System on a Chip (SoC) to perform the complex calculations required for such a task. The reconstruction problem, which MOCI can solve, contains many complex computer vision subroutines that can be used in less complicated computer vision pipelines.
Article
Full-text available
adiation effects in solid-state microelectronics can be split into two general categories: cumulative effects and single-event effects (SEEs). Cumulative effects produce gradual changes in the operational parameters of the devices, whereas ior in circuits. The space radiation environment provides a multitude of trapped, solar, and cosmic ray charged particles that cause such effects, interfere with space- system operation, and, in some cases, threaten the survival of such space systems. This article will describe these effects and how their impact may be mitigated in
Article
The Lunar Reconnaissance Orbiter (LRO) and Chandrayaan-1 both carry scientific investigations designed to measure the energetic particle fluxes and radiation dose in low lunar orbit. Mission operations overlapped for more than two months thus providing simultaneous measurements in low, polar lunar orbit. This was a time period of low solar activity and therefore the energetic radiation exposure was confined to the galactic cosmic rays (GCR) and albedo from the lunar surface created by interactions of the GCR with the lunar surface. The sensor basics and a survey of the observations will be presented.
Article
The radiation environment in low-Earth orbit is a complex mixture of galactic cosmic radiation, particles of trapped belts and secondary particles generated in both the spacecraft and Earth's atmosphere. Infrequently, solar energetic particles are injected into the Earth's magnetosphere and can penetrate into low-Earth orbiting spacecraft. In this paper, the sources of charged-particle radiation that contribute significantly to radiation exposure on manned spacecraft are reviewed briefly, and estimates of expected dose rate for the upcoming International Space Station that are based on measurements made on the Russian Mir orbital station are provided.
Thermal management and design of high heat small satellite payloads
  • versteeg
C. Versteeg, "Thermal management and design of high heat small satellite payloads," 32nd Annual AIAA/USU Conference on Small Satellites, 2018.
Body of knowledge for graphics processing units (gpus) nepp-bok-2018
  • E Wyrwas
E. Wyrwas, "Body of knowledge for graphics processing units (gpus) nepp-bok-2018," 2018. [Online]. Available: www.nasa.gov/sites/default/files/ atoms/files/2017-8-1 stip final-508ed.pdf
Standard materials and processes requirements for spacecraft
  • R R J Roe
R. R. J. Roe, "Standard materials and processes requirements for spacecraft," Technical Report NASA-STD-6016A, November 2016.