Secure Video Streaming Using Dedicated Hardware

Abstract

The purpose of this article is to present a system that enhances the security, efficiency, and reconfigurability of an Internet-of-Things (IoT) system used for surveillance and monitoring. A Multi-Processor System-On-Chip (MPSoC) composed of Central Processor Unit (CPU) and Field-Programmable Gate Array (FPGA) is proposed for increasing the security and the frame rate of a smart IoT edge device. The private encryption key is safely embedded in the FPGA unit to avoid being exposed in the Random Access Memory (RAM). This allows the edge device to securely store and authenticate the key, protecting the data transmitted from the same Integrated Circuit (IC). Additionally, the edge device can simultaneously publish and route a camera stream using a lightweight communication protocol, achieving a frame rate of 14 frames per Second (fps). The performance of the MPSoC is compared to a NVIDIA Jetson Nano (NJN) and a Raspberry Pi 4 (RPI4) and it is found that the RPI4 is the most cost-effective solution but with lower frame rate, the NJN is the fastest because it can achieve higher frame-rate but it is not secure, and the MPSoC is the optimal solution because it offers a balanced frame rate and it is secure because it never exposes the secure key into the memory. The proposed system successfully addresses the challenges of security, scalability, and efficiency in an IoT system used for surveillance and monitoring. The Rivest-Shamir-Adleman (RSA) encryption key is securely stored and authenticated, and the edge device is able to simultaneously publish and route a camera stream feed high-definition images at 14 fps. The proposed system enhances the security, efficiency, and reconfigurability of an IoT system used for surveillance and monitoring. The RSA encryption key is effectively protected by the implementation of an MPSoC with Programmable Logic (PL), which also allows for faster processing and data transmission.

Full text

Journal of Signal Processing Systems (2023) 95:1265–1275
https://doi.org/10.1007/s11265-023-01886-4
Vol.:(0123456789)
1 3
Secure Video Streaming Using Dedicated Hardware
NicholasMurray‑Hill1· LauraFontes1 · PedroMachado1 · IsiborKennedyIhianle1
© The Author(s) 2023
Abstract
The purpose of this article is to present a system that enhances the security, efficiency, and reconfigurability of an Internet-
of-Things (IoT) system used for surveillance and monitoring. A Multi-Processor System-On-Chip (MPSoC) composed of
Central Processor Unit (CPU) and Field-Programmable Gate Array (FPGA) is proposed for increasing the security and the
frame rate of a smart IoT edge device. The private encryption key is safely embedded in the FPGA unit to avoid being exposed
in the Random Access Memory (RAM). This allows the edge device to securely store and authenticate the key, protecting
the data transmitted from the same Integrated Circuit (IC). Additionally, the edge device can simultaneously publish and
route a camera stream using a lightweight communication protocol, achieving a frame rate of 14 frames per Second (fps).
The performance of the MPSoC is compared to a NVIDIA Jetson Nano (NJN) and a Raspberry Pi 4 (RPI4) and it is found
that the RPI4 is the most cost-effective solution but with lower frame rate, the NJN is the fastest because it can achieve
higher frame-rate but it is not secure, and the MPSoC is the optimal solution because it offers a balanced frame rate and it is
secure because it never exposes the secure key into the memory. The proposed system successfully addresses the challenges
of security, scalability, and efficiency in an IoT system used for surveillance and monitoring. The Rivest-Shamir-Adleman
(RSA) encryption key is securely stored and authenticated, and the edge device is able to simultaneously publish and route
a camera stream feed high-definition images at 14 fps. The proposed system enhances the security, efficiency, and recon-
figurability of an IoT system used for surveillance and monitoring. The RSA encryption key is effectively protected by the
implementation of an MPSoC with Programmable Logic (PL), which also allows for faster processing and data transmission.
Keywords IoT· FPGA· Graphics Processor Unit (GPU)· PYNQ· MPSoC· Security streaming· Edge computing
1 Introduction
Internet-of-Things (IoT) has become a central focus in the tech-
nology industry, with the development of smart devices that col-
lect and exchange data over the internet. These devices, includ-
ing sensors, actuators, and other IoT-enabled technologies, are
used for a variety of purposes such as analysis, processing, and
automation. It is estimated that there will be approximately 75
billion IoT devices in use by 2025 [1]. The use of IoT technol-
ogy has expanded into various fields including smart energy,
industrial factories, transportation, and home automation.
However, the widespread integration of IoT systems in
our daily lives also leads to an increase in the amount of
data being collected and exchanged over the internet, raising
concerns about scalability and the protection of sensitive and
private data. Many IoT systems rely on centralised archi-
tectures, which process and perform security operations on
the cloud [2]. These architectures have limitations such as
scalability, transaction speeds, interoperability, and privacy/
security [3], and may also be a single point of failure in the
event of a breach. Centralised servers that manage keys and
act as a single trust authority can pose a significant security
risk to other devices on the network, as these keys often form
the foundation of security systems, cryptography algorithms,
and device authentication/verification [2].
* Pedro Machado
pedro.machado@ntu.ac.uk
Nicholas Murray-Hill
n0821510@my.ntu.ac.uk
Laura Fontes
n1119003@my.ntu.ac.uk
Isibor Kennedy Ihianle
isibor.ihianle@ntu.ac.uk
1 Department ofComputer Science, School ofScience
andTechnology, Nottingham Trent University, ERD220,
Clifton Campus, NottinghamNG118NS, Nottinghamshire,
UK
Received: 16 January 2023 / Revised: 18 July 2023 / Accepted: 19 July 2023 / Published online: 15 August 2023
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1266 Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
To address these issues, distributed infrastructures have
emerged, allowing modern IoT edge devices to perform their
own processing and transmit data at the edge. These devices
feature robust security mechanisms for secure authentication
and secret key storage, which can be used for encryption in a
secure, hardware-enforced manner to improve end-device secu-
rity for IoT systems and data integrity [4]. Field-Programmable
Gate Array (FPGA) offer such security features through the use
of programmable logic and reconfigurability after manufactur-
ing. This reconfigurability allows devices to be updated in order
to keep up with the constantly evolving technology landscape
and emerging security threats.
The main contribution of this article is the development
and implementation of a secure, efficient, and reconfigur-
able surveillance and monitoring IoT system using dedicated
hardware. By utilising the Multi-Processor System-On-Chip
(MPSoC)’s Programmable Logic (PL) to securely store and
authenticate a symmetric key, the proposed system improves
the security and integrity of data transmitted from an edge
device. Additionally, the use of MPSoCs allow the edge
device to simultaneously publish and route a camera stream
using a lightweight communication protocol, achieving a
high capture rate. The proposed system addresses many of
the challenges faced by current IoT systems, including scal-
ability, security, and efficiency.
The remainder of this article is organised as follows: rel-
evant literature and related works to the proposed IoT sys-
tem proposed in this article is reviewed in Section2, the
methodology used to develop and implement the proposed
system is described in Section3, the results and analysis of
the proposed system are presented in Section4, and finally,
the conclusion and future work are discussed in Section5.
2 Related Work
The IoT is a technology that aims to provide an infrastruc-
ture for applications that can coordinate the interaction
of people, things, and systems for a specific purpose [5].
These applications do not necessarily have a universally
adopted standard, but the architectural model of an IoT sys-
tem typically consists of three main layers: the perception
layer, which uses sensors and microcontrollers to perceive
the physical environment; the communication or network
layer, which processes and transports data; and the applica-
tion layer, which uses the data to deliver application-specific
services to the user [1]. The communication/network layer
uses various technologies to package and transmit data due
to the processing and bandwidth restrictions of many IoT
devices. Hypertext Transfer Protocol (HTTP), which is
commonly used for communication between devices on the
internet, is not suitable for low-powered IoT devices due
to its fully connection-oriented architecture, large header
size, and latency [6, 7]. Established communication proto-
cols that are more suitable for these power and bandwidth-
restricted IoT requirements include Constrained Application
Protocol (CoAP), Message Queueing Telemetry Transport
(MQTT), and Extensive Messaging and Presence Protocol
(XMPP). Çorak etal. [8] evaluated and compared the per-
formance of these protocols in a real-world IoT testbed. The
metrics considered were packet creation time and packet
delivery speed to determine the delay differences. The study
found that XMPP had the worst performance due to its use
of Extensible Markup Language (XML) format, which
increased latency. MQTT and CoAP had similar overall
performance in terms of packet creation and transmission
time, but MQTT was found to be more optimised and stand-
ardised. In addition to these protocols, wireless technolo-
gies such as Low Range (LoRa), Low Range Wide Area
Network (LoRaWAN), and Low-Power Wide Area Network
(LPWAN) can be used to enable long-range and low-power
communications for IoT devices [9]. These wireless proto-
cols are designed to provide low-power, wide-area networks,
making them ideal for use cases where devices need to trans-
mit small amounts of data over long distances, such as in
agriculture, smart cities, and industrial applications [9]. van
der Westhuizen and Hancke [10] conducted a more in-depth
comparison between CoAP and MQTT to determine which
was the most suitable for use with constrained devices, spe-
cifically sensors. The comparison considered communica-
tion delay and network traffic. Both protocols were found
to be good choices for resource-constrained devices, with
similar performance and response times. However, the most
suitable protocol depended on the overall requirements of
the system. CoAP was found to be the optimal choice for
interfacing with business systems, due to its small average
packet sizes and minimal battery/data usage. MQTT, on the
other hand, was found to be the preferred solution for sys-
tems such as home automation and sensor networks, where
device heterogeneity is more pronounced. MQTT was easier
to con for new devices and had the most effective data flow
thanks to its publish/subscribe model and use of Quality of
Service (QoS).
2.1 Edge Computing
Edge computing is a network architecture that involves pro-
cessing sensory (e.g. visual data) data closer to the source,
rather than on the cloud [11]. This allows for fast process-
ing and efficient handling of data intensive operations in
real-world scenarios such as the IoT. While edge computing
can offer benefits for IoT systems, there are also limitations
in terms of security. Khan etal. [11, 12] found that further
development is needed in areas such as authentication and
access control, and that tamper-proof architectures may be
one solution to addressing these security issues. However,
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1267Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
securing large scale and time-critical IoT systems can also
be challenging due to the cost of methods such as encryp-
tion in terms of latency, energy consumption, and network
bandwidth [13]. Additionally, the heterogeneity of devices
that communicate across these networks without a well-
established protocol can also pose challenges [1]. Fortu-
nately, professionals in the field are working to overcome
these limitations and improve the safety and efficiency of
communication between IoT devices.
2.2 FPGA Technology
FPGAs are specialised hardware devices that have gained
popularity in the edge computing space due to their ability
to solve problems through reconfigurable hardware circuits.
These circuits can be described using Hardware Descrip-
tion Languages (HDL) such as Verilog and Very High
Speed Integrated Circuit Hardware Description Language
(VHDL), and are made up of various logic units such as
look-up tables, flip-flops, and multiplexers. FPGAs offer
several benefits for security, parallel computing, and flex-
ibility to update hardware designs after deployment [14–16].
They have also been advanced through the use of System-
on-Chip (SoC), which integrate programmable logic with
real-time processors. An example of this is the AMD-Xilinx
Zynq Ultrascale+ MPSoC1, which includes an Advanced
Reduced Instruction Set Computing Machine (ARM) CPU,
programmable logic, and units for graphics and video pro-
cessing. While FPGAs and SoCs have similarities with
microcontrollers, FPGAs offer advantages in physical and
cybersecurity through encrypted bitstreams and key loading
mechanisms, and can act as a Root of Trust (RoT) by holding
security private keys and critical algorithms. FPGAs also
show greater efficiency in processing algorithms for image
processing and video transcoding due to their parallel com-
puting capabilities.
While FPGAs offer significant advantages for IoT, they
are considered complex due to the low-level hardware
knowledge required, such as VHDL and Verilog. To address
this, FPGA vendors have been promoting the use of high-
level design flows and tools that allow for the creation of
Register Transfer Level (RTL) designs using high-level lan-
guages like C, C++, System C and Open Computer Lan-
guage (OpenCL). However, the question remains as to how
well these high-level designs compare to manually written
RTL designs in terms of optimisation. Guo etal. [17] dis-
cussed that while High Level Synthesis (HLS) may not be
as optimised as manually written RTL designs for complex
designs, the use of directives like loop unrolling and loop
merging and pipelining can significantly improve resource
utilisation, reduce latency, increase resource sharing, and
optimise logic for video processing algorithms. These find-
ings suggest that FPGA technology can be more accessible
to designers without strong low-level hardware knowledge,
while still maintaining good performance.
In summary, the reviewed articles have demonstrated the
various considerations and challenges faced in the design
and implementation of an IoT system. Communication
protocols such as MQTT and CoAP have been shown to
be effective in resource constrained environments, but the
choice between them ultimately depends on the specific
requirements of the system. Edge computing has the poten-
tial to improve the efficiency and security of IoT networks,
but also comes with its own limitations that require further
development. FPGA technology offers advanced security
and parallel processing capabilities for IoT, but can be com-
plex to implement. High level synthesis tools, such as the
AMD-Xilinx Vivado HLS2, have been shown to improve
the productivity and performance of FPGA designs for real
time image processing applications, but may not always be
as optimised as manually written designs. These findings
highlight the importance of carefully evaluating the various
technologies and approaches available for a particular IoT
system in order to ensure optimal performance and security.
3 Methods
The proposed IoT system utilises the Ultra96-V2 Devel-
opment Board (Ultra96) equipped with a powerful AMD-
Xilinx Zynq UltraScale+ MPSoC ZU3EG3 device as the
main processing system at the perception layer. The perfor-
mance of the Ultra96-V2 Development Board (Ultra96) was
compared to a NVIDIA Jetson Nano (NJN) and a Raspberry
Pi 4 (RPI4) under the same testing conditions.
To establish a fair comparison, each processing device
(i.e. Ultra96, RPI4 and NJN) runs an MQTT client to pub-
lish data from its connected Universal Serial Bus (USB)
webcam to an MQTT broker, which acts as an intermediary
to route the data to interested parties. The camera feed is
then displayed on a Node-RED4 dashboard at the applica-
tion layer for subscribers to view. The use of MQTT and the
Node-RED dashboard allows for efficient and flexible com-
munication and data management within the system. The
1 Available online, https:// www. xilinx. com/ produ cts/ silic on- devic es/
soc/ zynq- ultra scale- mpsoc. html, last accessed 07/01/2023
2 Available online, https:// www. xilinx. com/ suppo rt/ docum entat ion- navig
ation/ design- hubs/ dh0090- vitis- hls- hub. html, last accessed 07/01/2023
3 Available online, https:// www. xilinx. com/ conte nt/ dam/ xilinx/ imgs/
produ cts/ zynq/ zynq- eg- block. PNG, last accessed 07/01/2023
4 Available online, https:// noder ed. org/, last accessed 07/01/2023
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1268 Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
system also implements security measures, such as bitstream
authentication, to protect against potential attacks. Overall,
the proposed IoT system utilises a variety of technologies to
coordinate the interaction of people, things, and systems for
a specific purpose (see Fig.1).
The Avnet Ultra96, which is powered by an AMD-Xilinx
Zynq UltraScale+ MPSoC device that combines an ARM
processor and FPGA. The Ultra96 is energy efficient and
performs well due to designated processors being respon-
sible for specific tasks. The Processor System (PS) in the
AMD-Xilinx Zynq UltraScale+ MPSoC runs an ARM64v8
Linux environment for running a web server while also
interfacing with the programmable logic via the ARM
eXtensible Interface 4 (AXI4) for user authentication and
on-field reconfigurability. ARM64v8 is a version of the
ARM architecture that supports 64-bit instructions. It is
used in some 64-bit ARM processors, such as those used in
the Avnet Ultra96.
The NJN5 is a small, powerful computer designed for use
in image and video processing applications. It is powered by
a quad-core ARM Cortex-A57 CPU and a 128-core NVIDIA
Maxwell GPU, running on an ARM64v8 Linux environ-
ment. Programming the NJN is typically done using the
NVIDIA JetPack SDK (NJPSDK), which includes a Linux-
based development environment and a variety of software
libraries for working with the CPU and GPU. Open Com-
puter Vision (OpenCv) compiled with the CUDA library
was used to achieve maximum performance when processing
visual data.
The RPI46 is a low-cost, single-board computer designed
for educational and hobbyist use. It is powered by a Broad-
com BCM2711, quad-core Cortex-A72 (ARMv8) 64-bit
SoC @ 1.5GHz and runs on a Linux-based operating sys-
tem, typically Raspbian. OpenCv was used to achieve maxi-
mum performance when processing visual data. The RPI4
is known to be cost-effective solution but with lower frame
rate compared to other devices.
The IoT system was not initially configured with any
software or input sensors, so a USB webcam was connected
to the USB 3.0 Type A port to capture the camera feed.
The Micro-B upstream port was used to connect to a host
workstation on the same local network, although the board
also supports WiFi connectivity. The device was booted
using a Micro Secure Digital (uSD) card that loaded the
Python productivity for Zynq (PYNQ) framework. This
open-source framework allows developers to program AMD-
Xilinx UltraScale+ Zynq FPGA devices, such as the one
used in this device, using Python. Furthermore, PYNQ was
designed to be used in embedded systems and provides a set
of libraries, drivers and Jupyter notebooks to enable easy
programmability of FPGAs through high-level programming
languages like Python. This setup allows for the physical
connection and control of the camera feed, which can be
streamed in the proposed system.
3.1 PYNQ Framework
Booting the device required software to be loaded onto an
SD card, in order to leverage the security and parallel hard-
ware execution benefits of the Ultra92-V2 programmable
logic, the approach was to use the PYNQ framework ver-
sion 2.67. This platform features a Linux operating system,
along with the Python software package and a Jupyter8 web
server for developing solutions on the board for rapid on-
field development and reconfigurability over a network. This
image should be flashed onto a uSD card with a capacity of
at least 16GB and inserted into the board. Once powered on,
the board can be accessed by connecting a USB Micro-B
cable to a host PC or by setting up a WiFi connection on the
local network. The board is configured with the default IP
address of 192.168.3.1, which allows access to the locally
hosted Jupyter web server.
This revised statement provides more clarity by break-
ing up the original sentence into several shorter sentences.
It also provides more detail on how to access the board,
Figure 1 Architecture Diagram. The camera is connected to the
Processing System, with components 1) Ultra96, 2) NVIDIA Jetson
Nano (NJN), and 3) Raspberry Pi 4 (RPI4) streaming the video to an
MQTT topic. This streaming is achieved through a pipeline created
using Node-Red, and the video can be accessed via the endpoint if
the authentication is successful.
5 Available online, https:// devel oper. nvidia. com/ embed ded/ jetson-
nano- devel oper- kit, last accessed 15/01/2023
6 Available online, https:// www. raspb errypi. com/ produ cts/ raspb erry-
pi-4- model-b/, last accessed 15/01/2023
7 Available online, http:// www. pynq. io/, last accessed: 05/05/2022
8 Available online, https:// jupyt er. org/, last accessed: 05/05/2022
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1269Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
including both USB and WiFi options, and clarifies that the
Jupyter web server is hosted locally. Additionally, it uses the
passive voice as requested.
3.2 CUDA
To develop and execute various components which run on
the board, there are prerequisites that should be installed
during the setup phase. OpenCv version 4.5.19 for Python
is used to retrieve frames from an input device, such as a
webcam or IP camera. NumPy version 1.16.010 is also used
within the Jupyter notebook to manipulate data structures,
such as arrays. This was used to read and write user’s cre-
dential files. An MQTT broker should also be installed
to route the data between publisher and subscribers, so
the Mosquitto-MQTT version 1.4.1511 was installed from
Ubuntu’s open-source universe repository. Finally, to create
an MQTT client to publish the stream from the embedded
processing system, the Paho-MQTT12 Python package was
installed. At this point, all the prerequisites to develop and
execute the system are in place on the PYNQ Linux environ-
ment. A detailed list of the tools and equipment used in the
system can be found in Table1.
3.3 Designing theSecure Bitstream oftheUltra96
To secure the system, a bitstream file was created to provide
confidentiality through a 256-bit secret key and method of
authentication. The key is described in the FPGA logic and
is embedded within the FPGA unit, allowing the system
to securely store the private key and prevent it from being
exposed in the RAM. The system employs a pair of keys,
consisting of a public and a private key, to ensure secure
message authentication. Nevertheless, the private key is
securely stored in the FPGA logic, making the proposed
method safer than other authentication methods that store
private keys externally. The use of a private key stored in
the FPGA logic ensures that only authorised devices with
the corresponding public key are granted access to the sys-
tem and camera stream. This is achieved through a secure
authentication process where the authorised device sends
Table 1 List of devices and frameworks used in this work.
Resource Type Description
Anvet Ultra96 Equipment System on Chip device for authenticating user key and running the various components of
project
NJN Equipment Embedded system capable of achieving higher frame rates and running the various com-
ponents of project
RPI4 Equipment Low cost embedded system, equipped with 2GB of DDR, ideal for reducing the costs and
running the various components of project
Full HD 1080p Web Camera Equipment Camera used to capture environment for smart camera
Vivado HLS 2020.1 Development Tool AMD-Xilinx tool used for creating custom authentication Intellectual Property (IP) core
Vivado 2020.1 Development Tool AMD-Xilinx tool used for creating hardware design and generating the secure bitstream
PYNQ Framework Framework for booting the Ultra96 board with Ubuntu 18.04 and interfacing with the
programmable logic
OpenCV Library Library used to capture input streams, such as camera and online video stream
NVIDIA CUDA 11 Development Tool NVIDIA tool used for accelerating the video streaming using CUDA IP cores
NumPy Library Various array manipulation functionalities and for saving/reading the user credentials file
Paho MQTT Library Used to create an MQTT client on the Ultra96 to publish the camera stream to a topic
Mosquitto-MQTT Software Used to run a MQTT broker which listens to client publishing to manage and route to the
correct subscribers
Node-RED Development Tool The application layer is created using Node-RED to create a IoT flow
PuTTY Terminal Emulator An emulator which has built in protocols such as SSH to connect to the Ultra96
9 Available online, https:// opencv. org/, last accessed: 05/05/2022
10 Available online, https:// numpy. org/, last accessed: 05/05/2022
11 Available online, https:// mosqu itto. org/, last accessed: 05/05/2021
12 Available online, https:// pypi. org/ proje ct/ paho- mqtt/, last accessed:
05/05/2022
Figure2 Timing and latency summary. The estimated time is 2.88ns
with an uncertainty of 1.25ns, which is below the target clock of
10ns. In terms of latency, both the maximum and minimum values
are 34 clock cycles or 0.34us.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1270 Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
an authentication message encrypted with the public key.
The FPGA then decrypts the message using the private key
stored in its logic to confirm that the device is authorised
for providing an additional layer of security to the system,
making it more resistant to unauthorised accesses. The high
level design flow was used to develop the hardware design
at a higher level of abstraction using C/C++ code, which
was then converted into optimised RTL code by a compiler.
Custom Intellectual Property cores were also included in the
design and interfaced with via the PYNQ framework to run
on the Ultra96 programmable logic.
The process of building the authentication IP core begins
with using AMD-Xilinx Vivado HLS13 software. Using the
HLS software, the top level function was written in C con-
taining a secret key, authentication method to compare the
valid key with the input key, along with the required I/O
ports for the PL to interface with the IP block. In this case
there were two ports, one of which was the key with a size of
256-bits and the other being the authentication result which
was a single bit boolean value to represent whether the input
key was valid or not. Due to the size of these ports being
relatively small, the AXI4-Lite protocol was utilised for the
Ultra96 processing system to interface with the IP block,
as this is generally a suitable design choice for smaller data
transfers. To optimise the design, various loop and array
optimisation directives were tested in the hope to reduce
the estimated clock time and maximum clock cycles. By
using the pipeline Pragma in the loop to compare the keys,
the maximum clock cycles was reduced from 64 to 34. This
directive works by reducing the initiation interval for the
loop by allowing concurrent execution of the operations. To
verify the output of the top level function a test bench was
written, this test bench was used by the HLS tool during C
simulation, synthesis and C/RTL co-simulation to validate
that the produced RTL was functionally identical to the C
code that was written and therefore confirming that the IP is
working as intended to be packed and exported. The timing
and latency summary for the IP is presented in Fig.2.
To verify the functionality of the custom IP block, a test
bench was written in C. This test bench was used by the
HLS tool during C simulation, synthesis, and C/RTL co-
simulation to validate that the produced RTL was function-
ally identical to the C code, and therefore confirm that the
IP was working as intended. Once the custom IP block was
exported, it could be used as part of the wider system by
importing it into the AMD-Xilinx Vivado Design Suite.
This tool has an IP Integrator, which was used to build the
hardware design by integrating the custom IP block with IPs
available in the AMD-Xilinx’s IP catalogue. A block dia-
gram, shown in Fig.3, was generated using the Vivado Tool,
containing the Zynq UltraScale+ MPSoC block, which rep-
resents the processor of the Ultra96 and configures clocks,
peripherals, and other settings. To transfer the authentication
data between the PS and the custom IP, a single memory-
mapped ARM eXtensible Interface (AXI) master and AXI
slave Interconnect was included. The reset signals were han-
dled by the Processor System Reset IP block.
The custom IP block was integrated into the wider sys-
tem using the AMD-Xilinx Vivado Design Suite. The IP
Integrator tool was used to build the hardware design by
combining the created IP block with IPs available in the
AMD-Xilinx catalogue. The design was then simulated,
synthesised, and implemented to generate a bitstream. The
bitstream, .hwh file, and driver file were transferred to the
Ultra96 device to be imported using the PYNQ Overlay
class. This process allows for the custom IP block to be
used in the system to provide secure authentication. The
IP block is lightweight and only uses a small portion of the
programmable logic resources on the device.
Figure3 IP Integrator Block Design.
13 Available online, https:// www. xilinx. com/ produ cts/ design- tools/
vivado. html, last accessed: 05/05/2022
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1271Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
The IP block is lightweight and only uses a small por-
tion of the programmable logic resources on the device.
Figure4 shows the resource utilisation.
PYNQ is accessed through a local Jupyter web server
at 192.168.3.1. It allows the execution of Python packages
and libraries on the Ultra96 board. The PYNQ Overlay
class can be used to view and interface with the PL of the
Ultra96 using the previously created bitstream and default
overlay driver to access the IP’s ports configured in the
drivers file. The authentication result is retrieved using
three specific addresses: the start control signal (0x000),
the offset of the input key port (0x080), and the offset of
the data out port (0x100). The user’s symmetric key can
be loaded into the input key port a 4-byte integer at a time.
The start control signal is set to high to start the IP and
the authentication output is read from the data out port. If
the key is valid, the camera is initialised and published to
the MQTT broker.
The MQTT broker was configured to automatically
start on boot and run on localhost with the port 1883.
OpenCv and Paho-MQTT were also imported for use in
capturing and publishing the camera stream. A config-
uration file was placed on the device to define system
parameters such as camera settings, MQTT settings, and
the path to the credentials file. This file allows the user
to easily update system parameters without requiring
knowledge of the system. The data visualisation at the
application layer was the final component for the project.
This layer is responsible for connecting the clients or sub-
scribers to the MQTT broker and displaying the secured
camera feed. Node-RED is a browser-based editor, where
flows can be built using a catalogue of nodes to fit custom
IoT requirements. Additional nodes can also be installed
via node package manager. This tool was chosen for the
project due to being open source and high productivity,
where additional nodes can be quickly inserted into the
flow and deployed instantly. The flow that was designed
consisted of an MQTT input node, which is configured to
the MQTT broker running on the host workstation. This
node receives the message payload from the broker as a
base64 string and then passes this into an HTML image
template, which is finally connected to a dashboard widget
template where it is displayed automatically. Once this
flow is deployed, the dashboard can be accessed on the
local network at the URL 127.0.0.1:1880/ui. The designed
flow is shown in Fig.5.
Figure4 FPGA Resources
Utilisation. The figure demon-
strates that the authentication
logic occupies less than 2% of
the FPGA resources, indicat-
ing that it can be seamlessly
integrated with custom FPGA
designs without substantially
compromising significant
resources.
Figure5 Node-RED Flow. The
diagram illustrates the pipeline
that enables the decoupling of
the processing device from the
visualisation endpoint.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1272 Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
4 Results
The aim of this project was to build a flexible and reconfig-
urable edge device that could protect the integrity of data
within a surveillance and monitoring IoT system. To achieve
this, a secure authentication mechanism was implemented
to guarantee that the edge device could only publish the
camera stream when data integrity and authenticity could
be assured. This was accomplished by concealing a 256-bit
secret key and method of authentication inside a bitstream
file, which is the hardware description of an FPGA and is
difficult to reverse engineer due to being a stream of bits
that only describe the hardware logic itself. This provided
the necessary confidentiality to protect the key.
The proposed system was tested under the same conditions
on the Ultra96, NJN and RPI4 to ensure that each version of the
IoT device delivered the expected functionality and behaviour.
Several tests focused on the integration of the IP within the over-
all system were carried out to ensure that the system is working
as intended. This involved testing the end-to-end process of cap-
turing the camera stream, publishing the data over MQTT, and
displaying the stream on the dashboard. These tests were carried
out by setting up the Ultra96 board, NJN and RPI4 with the boot
image and prerequisites, importing the bitstream, and running
the python script to capture and publish the camera data. The
Node-RED flow was then set up, and the dashboard accessed
to verify that the stream was being displayed as expected on
all devices. Additionally, the system was tested under various
scenarios such as using the correct key, using an incorrect key,
and attempting to access the stream without providing a key, to
ensure that the system was functioning as intended and that the
authorisation process (see Fig.6) was secure (see Fig.6). These
tests were important to ensure that the Ultra96, NJN and RPI4
provide optimal solution, balanced frame rate and secure key
storage. It is clear from Table2 that the hardware implementa-
tion (i.e. Ultra96) had the same performance has the software
implementation (i.e. NJN and RPI4) but without exposing the
private key into the RAM.
Ultra96, the top level function was written in C, containing
the secret key and authentication method to compare the valid
key with the input key, along with the required I/O ports for the
PS to interface with the IP block. The AXI4-Lite protocol was
used for the Ultra96 processing system to interface with the IP
block, as it is suitable for smaller data transfers. To optimise
the design, various loop and array optimisation directives were
tested to reduce the estimated clock time and maximum clock
cycles. By using the pipeline Pragma in the loop to compare
the keys, the maximum clock cycles was reduced from 64 to 34.
To verify the output of the top level function, in the Ultra96,
a test bench was written and used by the HLS tool during C
simulation, synthesis, and C/RTL co-simulation to validate
that the produced RTL was functionally identical to the C code
and that the IP was working as intended. Once the custom
Figure6 Authentication Unit
Tests. The output of the seven
unit tests, where the first test
involved providing the correct
key followed by three tests with
incorrect keys, and the last three
tests involved two incomplete
keys.
Table 2 Authentication process (number of attempts) per each pro-
cessing system.
Tests RPI4 NJN Ultra96
Correct Key 10 10 10
Invalid Key 5 5 5
Incomplete Key 7 7 7
Empty Key 4 4 4
Wrong key 6 6 6
Successful authentications 10 10 10
Unsuccessful authentications 22 22 22
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1273Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
IP block was exported, it could be used as part of the wider
system by importing it into the AMD-Xilinx Vivado Design
Suite. This tool has an IP Integrator, which was used to build
the hardware design by integrating the created IP block with
IPs from the AMD-Xilinx’s IP catalogue. A single memory-
mapped AXI master and AXI slave Interconnect was included
to ensure that the system was able to handle various scenarios
that may occur during operation. These tests included dif-
ferent combinations of correct and incorrect keys, as well as
edge cases such as missing or incorrect bytes in the key. It was
essential that all of these tests passed in order to consider the
IP secure and fit for purpose. In addition to these unit tests, the
overall system was also tested to ensure that it functioned as
intended. This included testing the MQTT communication pro-
tocol, the PYNQ framework, and the Node-RED dashboard.
Overall, the testing of the system showed that the objective of
building a flexible and reconfigurable edge device was met, as
the system was able to securely store and use a secret key and
authentication method, and was also easily configurable and
adjustable through the use of a configuration file and various
software tools. A demo video [18] is available on Zenodo14
demonstrating the system working (Fig.7).
The final results obtained for all the devices are listed in
Table3.
The proposed system was tested under the same conditions
on the Ultra96, NJN and RPI4 to evaluate the performance and
security of the IoT device. The results showed that the NJN
achieved the highest frame rate of 30 fps (real-time), making it
the best in terms of frame rate. The RPI4 offered a more cost-
effective solution but with a lower frame rate of 6 fps, making
it the worst in terms of frame rate. And the Ultra96 achieved
a frame rate of 14 fps and offered a safer solution by securely
storing the RSA encryption key in the FPGA unit, making it
the best in terms of security and performance.
5 Discussion andFuture Work
In comparison to other authentication systems that rely
solely on the use of a CPU, the proposed IoT system utilis-
ing an FPGA has several advantages. One main advantage
is the improved security provided by storing the secret key
and authentication method within the FPGA bitstream, as
it is not accessible in a readable format outside the device.
This is in contrast to a CPU-only system where the secret
key and authentication method may be stored in plaintext or
encrypted in memory, which could potentially be accessed
by an attacker with the appropriate tools and knowledge.
On the CPU side, private keys are often stored in the file
$HOME/.ssh/id_rsa as plain text, which poses a security
risk. However, tools like valgrind15 and Hex editors can be
utilised to inspect processes loaded into memory, making
it possible to detect any potential breaches. Additionally,
one can enhance security by adding authorised public keys
to $HOME/.ssh/authorized_keys. Nevertheless, in the case
of FPGA logic, keeping authorised public and private keys
described within the logic makes it exceptionally challeng-
ing to breach security. This is because FPGAs are configured
with hardware-level security features that can help to prevent
unauthorised access and ensure that the keys remain secure.
Although there are risks associated with storing private keys
in plain text on the CPU side, there are also measures that
can be taken to mitigate these risks, and the use of FPGA
logic can provide an additional layer of security.
Another advantage of the proposed system is its recon-
figurability and flexibility. With the use of a configuration
file, the user is able to easily update the location of the bit-
stream, as well as change the camera input and MQTT set-
tings without requiring in-depth knowledge of the system.
This is not necessarily possible with a CPU-only system,
as changes to the system may require modifications to the
Figure7 Ultra96 CPU resource
consumption.
14 Available online, https:// youtu. be/ 8AXlf 6tRZyo, last accessed
07/01/2023 15 Available online, https:// valgr ind. org/, last accessed: 25/03/2023
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1274 Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
codebase and potentially require the expertise of a software
developer. Therefore, the user can decide which program-
ming system to use based on the budget, security and frame
rate restrictions. In terms of performance, the Ultra96 is able
to stream the camera feed at a maximum of 14 fps, which is
higher than the aim of 6 fps offered by the RPI4. This is due
to the efficient resource utilisation of the FPGA, as seen in
Fig.6 where the device is only utilising 63.6% of its CPU
resources. In comparison, a CPU-only system may struggle
to handle the processing demands of the camera stream-
ing and MQTT communication simultaneously, potentially
resulting in lower frame rates or slower performance.
In summary, the proposed IoT system utilising an FPGA
for authentication offers improved security, reconfigurabil-
ity, and performance compared to systems that rely solely
on a CPU. There are many directions in which future work
on this project could go. One possible avenue of research is
to improve the security of the system by implementing more
advanced forms of authentication. For example, instead of
using a simple symmetric key, a more secure method such as
a public-private key pair could be used. This would require
the use of a cryptographic accelerator or hardware security
module to ensure that the key operations can be performed
quickly and efficiently on the edge device. Another possi-
bility is to incorporate additional security measures to pro-
tect against physical tampering with the device. This could
include the use of tamper-evident seals or hardware-based
intrusion detection to alert the user if the device has been
opened or tampered with.
Another area for improvement could be to optimise the
system for better resource utilisation. This could involve
using more advanced optimisation techniques during the
design phase, or implementing more efficient protocols for
communication between the different components of the sys-
tem. Finally, it would be interesting to explore the possibility
of implementing machine learning algorithms on the edge
device to enable more advanced forms of data analysis and
decision-making. This could involve training a model on the
device to identify certain patterns or characteristics in the
data, and then using this model to make decisions about how
to handle the data. Overall, there are many exciting direc-
tions in which this project could be taken, and we believe
that it has the potential to make a significant impact in the
field of edge computing and IoT security.
Acknowledgements The authors would like to express their gratitude
to Mr Flemming Christensen and Sundance Multiprocessor Technology
for their invaluable support and assistance in the form of AMD-Xilinx
training.
Data Availability To request access to the source code, please contact
the corresponding author.
Declarations
Conflict of Interest The manuscript does not have any associated data,
and the source code is available upon request. For more information
regarding the work, please reach out to the corresponding author, who
will gladly provide additional details.
Open Access This article is licensed under a Creative Commons Attri-
bution 4.0 International License, which permits use, sharing, adapta-
tion, distribution and reproduction in any medium or format, as long
as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons licence, and indicate if changes
were made. The images or other third party material in this article are
included in the article's Creative Commons licence, unless indicated
otherwise in a credit line to the material. If material is not included in
the article's Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will
need to obtain permission directly from the copyright holder. To view a
copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.
References
1. Nord, J. H., Koohang, A., & Paliszkiewicz, J. (2019). The internet
of things: Review and theoretical framework. Expert Systems with
Applications, 133, 97–108.
2. Kouicem, D. E., Bouabdallah, A., & Lakhlef, H. (2018). Internet
of things security: A top-down survey. Computer Networks, 141,
199–221.
3. Farahani, B., Firouzi, F., & Luecking, M. (2021). The convergence
of iot and distributed ledger technologies (dlt): Opportunities,
Table 3 Results comparison.
a Available online, https:// uk. farne ll. com/ avnet/ aes- ultra 96- v2-g/ sbc- arm- cortex- a53- cortex- r5/ dp/ 30504 81, last accessed 15/01/2023
b Available online, https:// thepi hut. com/ produ cts/ raspb erry- pi-4- model-b? varia nt= 20064 05267 4622, last accessed 15/01/2023
c Available online, https:// thepi hut. com/ produ cts/ nvidia- jetson- nano- 2gb- devel oper- kit, last accessed 15/01/2023
Device Security aspects Frame rate no auth Frame rate with auth Image size Average price
Ultra96 Safest because the private key is 14 14
1920 ×1080
£300.29 GBPa
never exposed to memory. near real-time near real-time
RPI4 Unsafe because the secure key was 6 7
1920 ×1080
£52.00 GBPb
exposed to the memory. far real-time far real-time
NJN Unsafe because the secure key was 30 30
1920 ×1080
£61.00 GBPc
exposed to the memory. real-time real-time
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1275Journal of Signal Processing Systems (2023) 95:1265–1275
1 3
challenges, and solutions. Journal of Network and Computer
Applications, 177, 102936.
4. Johnson, A. P., Chakraborty, R. S., & Mukhopadhyay, D. (2015).
A puf-enabled secure architecture for fpga-based iot applications.
IEEE Transactions on Multi-Scale Computing Systems, 1(2),
110–122.
5. Wortmann, F., & Flüchter, K. (2015). Internet of things. Busi-
ness & Information Systems Engineering, v. 57. Available online:
https:// link. sprin ger. com/ conte nt/ pdf/ 10. 1007/ s12599- 015- 0383-
3. pdf. Accessed 1 July 2023.
6. Patel, C., & Doshi, N. (2020). A novel mqtt security framework in
generic iot model. Procedia Computer Science, 171, 1399–1408.
7 . Shah, S. W. H., Mian, A. N., Mumtaz, S., Wen, M., Hong, T., & Kadoch,
M. (2019). Protocol stack perspective for low latency and massive
connectivity in future cellular networks. In: ICC 2019-2019 IEEE
International Conference on Communications (ICC), pp. 1–7. IEEE.
8. Çorak, B. H., Okay, F. Y., Güzel, M., Murt, Ş., & Ozdemir, S.
(2018). Comparative analysis of iot communication protocols.
In: 2018 International Symposium on Networks, Computers and
Communications (ISNCC), pp. 1–6. IEEE.
9. Queralta, J. P., Gia, T. N., Zou, Z., Tenhunen, H., & Westerlund,
T. (2019). Comparative study of lpwan technologies on unli-
censed bands for m2m communication in the iot: Beyond Lora
and Lorawan. Procedia Computer Science, 155, 343–350.
10. vander Westhuizen, H. W., & Hancke, G. P. (2018). Practical
comparison between coap and mqtt-sensor to server level. In:
2018 Wireless Advanced (WiAd), pp. 1–6. IEEE.
11. Khan, W., Ahmed, E., Hakak, S., Yaqoob, I., & Ahmed, A. (2019).
Edge computing: A survey, future generation computer systems.
12. Premsankar, G., Di Francesco, M., & Taleb, T. (2018). Edge com-
puting for the internet of things: A case study. IEEE Internet of
Things Journal, 5(2), 1275–1284.
13. Mohanty, S. N., Ramya, K., Rani, S. S., Gupta, D., Shankar, K.,
Lakshmanaprabu, S., & Khanna, A. (2020). An efficient light-
weight integrated blockchain (elib) model for iot security and
privacy. Future Generation Computer Systems, 102, 1027–1037.
14. Elnawawy, M., Farhan, A., AlNabulsi, A., Al-Ali, A. -R., &
Sagahyroon, A. (2019). Role of fpga in internet of things applica-
tions. In: 2019 IEEE International Symposium on Signal Process-
ing and Information Technology (ISSPIT), pp. 1–6. IEEE.
15. Teubner, J., & Woods, L. (2013). Data processing on fpgas. Syn-
thesis Lectures on Data Management, 5(2), 1–118.
16. Hoozemans, J., Peltenburg, J., Nonnemacher, F., Hadnagy, A.,
Al-Ars, Z., & Hofstee, H. P. (2021). Fpga acceleration for big
data analytics: Challenges and opportunities. IEEE Circuits and
Systems Magazine, 21(2), 30–47.
17. Guo, K., Zeng, S., Yu, J., Wang, Y., & Yang, H. (2017). A survey
of fpga-based neural network accelerator. arXiv preprint arXiv:
1712. 08934
18. Murray-Hill, N., & Machado, P. (2023). Secure video streaming using
dedicated hardwar. Zenodo. https:// doi. org/ 10. 5281/ zenodo. 75131 29
Publisher's Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com