Security implications of SCADA ICS virtualization: survey and future trends

Full text

Security implications of SCADA ICS virtualization: survey and future trends
T. Cruz, R. Queiroz, P. Simões, E. Monteiro
University of Coimbra, Portugal
{tjcruz, rqueiroz, psimoes, edmundo}@dei.uc.pt
Abstract: In recent years, Supervisory Control and Data Acquisition (SCADA) Industrial Control
Systems (ICS) – a kind of systems used for controlling industrial processes, power plants or assembly
lines – have become a serious concern because of security and manageability issues. Years of air-
gaped isolation, the increased coupling of ICS and Information and Communication Technology (ICT)
systems, together with the absence of proper management and security policies, disclosed several
weaknesses in SCADA ICS. Suddenly, these systems were faced with a reality that was familiar for
ICT infrastructure managers for decades, which has driven the need for the development of specific
technologies, as well as the establishment of management frameworks and the adoption of security-
oriented policies. Virtualization was one of such developments, whose influence spawns several
domains, from networking and communications to mass storage and computing resources.
For ICT, the rise of virtualization constituted a paradigm shift, with significant gains in terms of
resource consolidation, manageability or even security. These benefits are yet to fully reach the ICS
domain, despite recent developments geared towards the introduction of hypervisors or software-
defined networking within such systems. This paper provides an overview on the usage of such
technologies to improve SCADA ICS security and reliability also proposing advanced use cases.
Keywords: Virtualization, Critical Infrastructure Protection, Industrial Control Systems.
1. Introduction
In recent years, SCADA ICS – a kind of systems used for controlling power plants, assembly lines or
industrial processes, often part of critical and/or strategic infrastructures – have become a serious
concern because of security and manageability issues. After years of air-gaped isolation, the
increased coupling of ICS and ICT systems, together with the absence of proper management and
security policies (Krutz 2006), disclosed several weaknesses in SCADA ICS, which were left exposed
to attacks, with potentially catastrophic consequences. Nevertheless, these problems hardly
constitute any novelty within the ICT domain, which has dealt with them for decades, driving the need
for the development of specific tools and protocols, as well as the establishment of management
frameworks, such as Information Technology Infrastructure Library (ITIL) change management
(Gallup 2009) or security oriented policies.
However, ICT-specific practices cannot be easily ported to the ICS domain. For ICS operators,
equipment manufacturers and software developers alike, reliability is top priority. Continuous
operation and operational safety targets make it difficult to deploy several ICT-specific strategies and
tools, because of the potential impact on the ICS. This has pushed the industry, researchers and
standardization organizations to conceive ICS-specific security and management solutions and
frameworks, as well as publishing guidelines and guides documenting best practices. New product
lines were also introduced, with added security features and management capabilities.
Still, the ICS paradigm itself remained relatively unchanged, as proposed solutions try to fix what is
wrong without attempting to introduce significant change into existing systems. This solution is far
from optimal, as typical lifecycle management operations such as security patch deployment are still
an issue in modern SCADA ICS, the same being true for change management. In contrast, these
issues have been addressed in the ICT domain for years, through the continuous development of
technologies, tools and practices, designed to address such needs. Virtualization technologies are
among these developments, which influence ICT computing and communications infrastructures.
Developments such as hypervisors, Software-Defined Networking (SDN) or Network Function
Virtualization (NFV) are reshaping the ICT ecosystem, providing the means to rationalize the use of

computing and communications resources, also being instrumental to optimize and/or improve
aspects such as lifecycle management, energy efficiency, reliability or security, among others.
From an ICS security and reliability perspective, device and infrastructure virtualization may have a
similar impact as they had for ICT, as the industry slowly starts to absorb some of the technologies,
customized and fine-tuned for critical infrastructure environments. However, this is a process
undergoing its early stages, not only because the specific ICS use cases for several virtualization
technologies have yet to be developed, but also because extensive testing is required for its
certification in such environments. In this scope, this paper analyses the application of virtualization
technologies for communications and computing resources in ICS contexts, with a focus on recent
developments, open challenges and benefits, from a security and reliability-oriented perspective.
The rest of this paper is structured as follows. Section 2 discusses the problem of security in
ICS/SCADA, also explaining the potential benefits of introducing domain-aware virtualization
technologies in such environments. Section 3 discusses the introduction of network virtualization
technologies in SCADA ICS and its security benefits. Section 4 addresses the advantages of
introducing partitioning hypervisors in ICS, describing a virtualized Programmable Logic Controller
(PLC) use case. Finally, section 5 presents conclusions insights about future developments.
2. Virtualization and SCADA ICS security
As their scope was originally restricted to isolated environments, SCADA systems were considered
relatively safe from external intrusion. However, as architectures evolved, these systems started to
assimilate technologies from the ICT world, such as TCP/IP and Ethernet networking. This trend,
together with the increasing adoption of open, documented protocols, exposed serious weaknesses in
SCADA architectures, a situation that was aggravated by factors like the use of insecure protocols,
such as Modbus (Triangle 2002) or inadequate product lifecycle management procedures (Igure
2006), the latter being responsible for the proliferation of devices and components beyond their end-
of-life support status. Also, the interconnection of the ICS network with organizational ICT network
infrastructures, and even with the exterior (for example, for remote management) brought a new wave
of security incidents, with externally initiated attacks on ICS systems increasing significantly,
especially when compared with internal attacks (Kang 2011). Overall, this situation has become the
root cause of many well-known ICS security incidents, such as the Stuxnet Trojan (O’Murchu 2011).
In fact, ICS security cannot be approached in the same way as its ICT counterpart, as both domains
differ significantly on their fundamental design principles. Due to its critical nature, ICS operation and
design practices frequently privilege availability and reliability over confidentiality and data integrity – a
perspective that is quite the opposite from the ICT philosophy, which follows an inverse order of
priorities (ISA-99.00.01).
The differences between the ICT and ICS domains also mean that there is no “one size fits all”
solution when it comes to choose and implement security mechanisms. The fundamental premises for
ICT security tools and commonplace lifecycle management procedures, such as patching and
updating a system, can become troublesome in an ICS, when faced with situations such as the
impediment / high cost of stopping production (Zhu 2011), or even the explicit prohibition by the
system’s manufacturer, as any software release has to be certified before being released. Also,
several security mechanisms, such as anti-virus software are frequently unadvised by SCADA
software providers, as they might interfere with the response latency of the host. The same rationale
applies to anything deployed in the middle of the critical communications path (e.g., an inline network
Intrusion Detection System), as it may induce latency or some other sort of reliability issue.
Ironically, much of the problems faced by ICS are not entirely new, as they were known well before in
the ICT domain, which has undergone several paradigm shifts and major technological steps to deal
with them. More recently, the rise the virtualization paradigm has become instrumental in changing
the ICT computing landscape, providing the means to leverage computing and communications

resources, through consolidation and efficient management. Technologies such as hypervisors, SDN
or NFV are contributing to rationalize, streamline and reshape infrastructures and devices, up to the
point of changing the way communications and computing resources are consumed by end-users.
In terms of security and reliability, the impact is manifold. For instance, by creating a virtual machine
(VM) snapshot it is possible to rollback changes in case of failure or corruption caused by a failed OS
patch or malicious tampering; VMs can be cloned for sandboxed testing, prior to deployment into
production; hypervisors can perform in-place behavior monitoring of instances for security and safety
purposes. Similarly, technologies such as SDN, which constitute a flow-oriented virtualization
mechanism for networks, allow for the flexible creation and management of network overlays on top
of existing physical infrastructures, while also enabling significant security and reliability benefits
(Proença 2015). NFV, in its turn, can work together with SDN to virtualize network equipment
functionality, spreading it across the communications and computing infrastructure in an efficient and
rational way, also enabling the creation of innovative security solutions designed to better couple with
the increasingly distributed nature of modern ICS and associated threats (Cruz 2015).
But the introduction of ICT-like virtualization techniques in ICS is not a straightforward process. For
operators, equipment manufacturers and software developers alike, reliability, operational safety and
continuous operation are top priorities, a situation that makes it difficult to deploy several IT-specific
strategies and tools, because of the potential impact on the ICS. For example, the latency overhead
of certain mechanisms may not be compatible with real-time operation requirements. Hypervisors
must cope with the (soft) real-time requirements of ICS applications; any attempt to introduce SDN or
NFV must account for the potential impact in terms of ICS reliability or latency.
Despite the constraints, the potential efficiency, security and reliability benefits for ICS are enough to
justify the progressive development and introduction of domain-aware virtualization technologies. For
instance, real-time hypervisors can provide safe partitioning and isolation, enabling the creation of
managed execution environments for real-time workloads, with continuous assessment of partition
behavior, also providing rollback capabilities for potentially compromised systems. Use of SDN
technologies can provide the ICS operator with the means to monitor the ICS communications
infrastructure behavior, while easing the implementation of countermeasures and deployment of
security mechanisms. As ICS become increasingly distributed, NFV can provide the means to
efficiently spread functional security components across the ICS communications and computing
infrastructure, in order to better couple with the dispersed nature of the protected systems. The next
two chapters will discuss how domain-aware virtualization can provide effective security benefits for
ICS, with a focus on two major scopes: communications and computing.
3. Virtualization of SCADA ICS communications infrastructures
This chapter is specifically concerned with the introduction of SDN and NFV technologies within the
SCADA ICS scope. For this purpose, the security benefits of the technologies hereby discussed will
be analyzed from a broad perspective, both in terms of the physical ICS dimension and dispersion of
its scope, ranging from plant-level to distributed Industrial Automation and Control Systems (IACS)
use cases. All sections will start with a brief introduction of its respective cornerstone concepts,
namely SDN and NFV, in order to ease its introduction in the context of SCADA ICS security.
3.1. SDN and SCADA ICS
SDN is an architecture that decouples forwarding functions (data plane) and network control (control
plane), with the aim of introducing direct programmability into the network, to applications and policy
engines alike. With SDN, packet forwarding is flow oriented, meaning both origin and destination are
taken into account, instead of just packet destination, as in traditional networking. Flow policies are
granted by an SDN controller, which manages the policies for a range of forwarding elements in a
given network, effectively moving control plane functions outside of the devices. Thus, SDN-capable
elements can be dynamically reconfigured over the network accordingly with the needs of network
services and applications. For this reason, the controller will have a broader view of the domain,

contrasting with the narrow view that an individual forwarding element has in a traditional IP network.
There are several SDN protocols, among which OpenFlow (ONF) is one of the most popular.
SDN allows for increased network flexibility and programmability, in particular for complex scenarios,
which benefit from the reduced overhead for management operations such as topology changes for
implementing overlay networks. Besides these benefits, SDN can also provide an effective
mechanism for security applications (Proença 2015). This is due to the fact that a centralized element
with a global view of all the network entities – such as devices, flows and network elements – is able
to provide more efficient information gathering and security reaction mechanisms, especially when
compared with the narrow local view individually provided by each forwarding element in traditional IP
networks. Moreover, flow-based forwarding can be used to increase the efficiency of a reaction, being
used to isolate or divert flows, instead of simply blocking an attack. This is useful to improve existing
security techniques – for example, allowing to dynamically divert attackers to honeypot systems, as
soon they are detected. SDN can also help handling Denial of Service (DoS) and Distributed DoS
(DDoS) attacks, by improving detection and reaction mechanisms.
Besides the generic security application scenarios, there have been several developments regarding
SDN-based security mechanisms for ICS. For instance, (Dong 2015) proposes reinforcing the
resilience of SCADA networks used for smart grid applications using a solution relying on three
elements (SCADA master, SDN controller, Intrusion Detection System – IDS), which coordinate with
each other in order to detect attacks and reconfigure the network so as to mitigate and overcome
identified problems. Suggested use cases include the dynamic establishment of routes to transmit
control commands only when necessary (to shorten the time window for tampering attempts);
automatic rerouting or dropping of suspicious packets to avoid spoofing or flooding attacks from
compromised SCADA elements; or the implementation of network monitors to deal with delay attacks.
(Irfan 2015), proposes using SDN for dynamic creation of virtual networks in order to isolate distinct
traffic and hosts, enabling traffic prioritization and secure partitioning. The concept is demonstrated
using an SDN controller proxy to create three isolated networks, which share the same physical
infrastructure, but have their own SDN controllers. Authors discuss the use of this architecture to
improve aspects such as authentication, confidentiality, integrity, non-repudiation and availability. A
similar approach is also suggested by (Machii 2015) as a way to minimize the attack surface, by using
SDN to dynamically segregate fixed functional groups within the ICS. A dynamic zone-based
approach is also proposed, taking advantage of the information obtained from field devices to
estimate the operation phase of the ICS (as each phase, such as start-up, normal operation or load-
change exhibit different behavior and communications profiles) and calculate the optimal zone
topology, deploying the needed SDN configuration in runtime. This strategy reduces the time and
spatial exposure to attacks, also providing the means to isolate compromised devices.
Also related to dynamic configuration techniques, (Chavez 2015) presents a security solution based
on network randomization, complemented with an IDS capable with near real time reaction
capabilities. This network randomization approach assigns new addresses to network devices in a
periodic basis or by request, in order to protect them against attacks that rely on knowledge about the
ICS topology (such as static device addresses). The responsible controller application keeps an
updated database of all the network specifications (mostly devices and real addresses), generating
overlay IP addresses for the same devices and for each flow, which are used to define the OpenFlow
rules on flow tables. This way, all the traffic flowing on the network uses “fake” overlay addresses that
are periodically randomized, reducing their useful lifetime and, consequently, the time window
available for any attacker to take advantage of that knowledge. The proposed IDS takes advantage of
the predictable, auto-similar, traffic patterns of ICS networks for identify attacks and trigger defense
reactions (a network randomization request, which will render useless any ongoing attack using old
overlay addresses). Attack detection makes use of machine learning algorithms and mathematical
methods, fed and trained using OpenFlow’s statistical counters.

(Silva 2015) also describes a dynamic technique that makes use of SDN to prevent eavesdropping on
SCADA networks. The intended goal is to deter attackers from collecting sequential data, which is
essential for breaking encryption, identify patterns and retrieve useful information from the payload.
By taking advantage of redundant network connectivity, a multi-path routing mechanism enables a
flow to be transmitted and split over different paths (see Figure 1) by resorting to an algorithm that
calculates the shortest path between two devices, dynamically assigns a cost to each one and uses
an OpenFlow timer (hard timeout) to periodically reinstall new flow rules.
Figure 1: Multi-flow, redundant routing for flow splitting (adapted from (Silva 2015))
(Genge 2016) proposes two distinct SDN-based techniques to mitigate and block ICS cyber attacks.
The first technique (see Figure 2), designed for single-domain networks, attempts to mitigate DoS
attacks by rerouting traffic, using information from the SDN controller. SDN controllers feed an
application that continuously monitors the state of the network links and communicates with the
controller to issue flow reconfiguration operations. Once an attack is detected (few details are
provided about this, though), the corresponding data flows are rerouted, in order to protect the ICS.
Figure 2: A single-domain SDN-based security solution (adapted from (Genge 2016))
The second technique (see Figure 3) targets multi-domain networks, with the goal of blocking the
attack as close as possible to the entry point in the network.
Figure 3: A multiple-domain SDN-based security solution (adapted from (Genge 2016))

For such a multi-domain network, each domain has its own OpenFlow controller, connected to a
centralized security application. This application receives information from the SDN controllers, having
access to a global perspective about the network – once an attack is detected, it will backtrack
towards its origin, by recursively issuing queries about the related flows to identify the previously
paired nodes, until the original network entrance point is found.
3.2. Network Function Virtualization and Distributed ICS
NFV is the result of the convergence between telecommunications infrastructures and infrastructure
virtualization. As network applications and services scale and evolve (not only in sheer capacity
requirements, but also in complexity), they imposed an added burden to the supporting
telecommunications provider infrastructure, requiring the use of specific network management and
traffic policies that cannot be provided by the network. In this perspective, NFV (Chiosi 2012) is a
significant development as it enables the creation of flexible and on-demand network services through
a service chain-based composition mechanism that uses network functions implemented in VNF
(Virtualized Network Functions) components comprising functionality such as NAT, IDS, Firewalls or
other service modules, implemented as VM appliances.
The NFV vision attempts to decouple network capacity from functionality, by conceiving an end-to-end
service as an entity that can be modeled and described by means of network function forwarding
graphs (Figure 4) involving interconnected VNFs and endpoints (also known as service chaining.
Figure 4: NFV Forwarding Graph example
This approach allows for creation of differentiated end-to-end services that can be provided by the
(ordered) combination of elementary VNF or physical functions, chained together by a Forwarding
Graph, which models the service flows (see Figure 5). Furthermore, VNF FGs can be nested to define
complex functions. VNFs are implemented in software, being interconnected through the logical links
that are part of a virtualized network overlay, which can be implemented using SDN.
Figure 5: NFV end-to-end service with VNFs (adapted from (Ersue 2013))
Eventually, even Physical Network Functions (conventional network devices with close coupled
software and hardware that perform network functions) can be involved in a Network Forwarding
Graph service chain (the concept of service chain is not exclusive of NFV). A virtualization layer
abstracts the physical resources (computing, storage, and networking) on top of which the VNFs are
deployed and implemented, with the supporting NFV Infrastructure (NFVI) being spread across
different physical locations, called Points of Presence (NFVI PoPs), as shown in Figure 5.
VNF-FG
SaaS
VNF 2 VM
Load
Balancing
VNF 3 VM
Firewall
VNF 4 VM
Policy and
Licensing
VNF 1 VM
IPS
PNF
Edge Router
NFV Infrastructure Points of Presence
(PoP – physical locations where hardware resources are deployed; PL – physical links)
PL
PL
PL
VNF-FG VNF-FG 2
(nested forwarding graph)
VNF 1
VNF 2C
VNF 3
LL
LL
LL
LL LL
Endpoint
LL LL
PL
PL
PL
Virtualization Layer
NFVI
POP
PL
NFVI
POP
NFVI
POP
NFVI
POP
Endpoint
VNF 2A VNF 2B
End-to-end network service
Legend
Virtualization
LL Logical Link

NFV$as$an$enabler$for$a$new$generation$of$distributed$IACS$
Use cases such as Internet of Things (IoT), wire to water generation, micro generation, smart
metering or smart water management constitute a new generation of distributed IACS that can only
be supported with the help of a complex distributed software stack, potentially also requiring the
involvement of third-parties, such as telecommunications and cloud operator infrastructures – for this
reason, the introduction of Network Function Virtualization component appliances, distributed across
geographically dispersed infrastructure PoPs, makes entire sense,.
As the IACS enters the customer premises, the NFV service abstraction model (services as
composition of VNFs) provides an effective way to introduce support components along the service
path – for instance, a data collection and analysis VNF can be added to the customer service chain
(eventually within a virtual Business Gateway service abstraction) to provide data collection for smart
metering scenarios. The same rationale applies for security purposes, as cyber-physical protection
(for example, to implement bump-in-the-wire encryption) or security anomaly detection VNFs can be
integrated within service chains, also using SDN to create flexible security monitoring and reaction
capabilities. Moreover, Distributed IDS (DIDS) components may be consolidated in the form of VNFs
optimally deployed in order to reduce service overhead and rationalize resources. For instance, the
DIDS components might be deployed in the form of VNFs, either shared among several Business
Gateway FGs or used exclusively by a service instance (Cruz 2015). Some manufacturers (RAD
2015) (ECI 2015) are starting to propose NFV products for ICS applications that implement this
philosophy, NFV capabilities in access nodes for optical transport or packet switched networks, for
hosting firewall, encryption or traffic monitoring VNFs
NFV is also an enabler for fog computing (or “edge computing”) scenarios, allowing parts of the
infrastructure to be deployed on the network edges, using virtualized platforms located between end
user devices and the cloud data centers. This approach addresses the need to process large data
streams in real time while working within the limits of available bandwidth, by placing some of
transactions and resources at the edge of the cloud (in locations close to end users), thus improving
the efficiency of the infrastructure by offloading processing tasks before passing it to the cloud. For
these reasons, fog computing is becoming a cornerstone concept for distributed IACS architectures,
providing a way to deal with the information volume generated by sensor streams in an efficient way.
The NFV paradigm is naturally compatible with fundamental premises for implementation of fog
computing distributed topologies. As such, it is envisioned that distributed awareness and IACS
cyber-security detection capabilities will take advantage of the NFV paradigm to support its underlying
deployment model, departing from the conventional, self-contained model and moving towards an
architecture capable of keeping up with the geographically dispersed nature of IoT IACS. Also, the
VNF deployment criteria may consider the availability of specific capabilities (such as raw processing
capacity) in a specific NFVI POP – for instance, per-subscriber security event processing components
may be hosted in a different NFVI POP from the one(s) hosting other VNFs for the DIDS service.
4. Real-time Hypervisors + SDN = towards a virtualized PLC
Born in the mainframe era, Virtual Machine Monitors (also called Hypervisors) have ultimately evolved
towards being supported in open, Commercial Off-the-shelf (COTS) hardware platforms. Specifically,
type-1 (bare metal) hypervisors have become popular in large-scale virtualization scenarios such as
datacenters, bringing several benefits in terms of resource consolidation, business continuity,
scalability, management and security.
But most type-1 hypervisors are optimized for ICT loads, being unsuitable for several ICS application
use cases, mostly due to the overhead of the mediation and translation mechanisms abstracting the
host hardware from the VM. This situation gradually began to change, as some operators started
virtualizing hosts with services deployed on general-purpose OS, such as SCADA Master Stations
(MS), Human-Machine Interfaces (HMI) or Historian Database servers (HDB), using conventional

type-1 hypervisors. This was possible due to the development of hardware-assisted memory
management and I/O mechanisms to implement robust resource affinity and reservation (such as VT-
d and PCI SRV-IO (Garcia-Valls 2014) support), providing performance guarantees while avoiding the
effect of resource overprovisioning.
Other ICS elements, such as process control devices can also potentially benefit from virtualization
technologies. For instance, (Cahn 2013) proposed the virtualization of Intelligent Electronic Devices
(IEDs) used to collect information from sensors and power equipment, with the purpose of optimizing
the maintenance and cost overheads, while increasing reliability. The same rationale could be applied
to Programmable Logic Controllers (PLC) devices, which constitute the focus of this section.
PLCs are pervasive devices in ICS, such as SCADA systems, being designed to control industrial
processes. Contemporary PLCs are the outcome of an evolutionary process that started with the first
generation of relay-based devices, progressively incorporating technologies such as microprocessors,
microcontrollers and communications capabilities, ranging from serial point-to-point or bus topologies
to Ethernet and TCP/IP. Despite modern PLCs often being embedded devices with commodity
Instruction Set Architecture (ISA) System-on-Chip or CPUs (PowerPC, x86 or ARM), running Real-
Time Operating Systems (RTOS), its virtualization was not deemed feasible until recently, due to the
lack of specific hardware, software and infrastructure support.
4.1. Towards the virtual PLC
PLCs are designed for reduced and deterministic latency, operating under strict timing constrains that
are dependent on factors such as the end-to-end and event response latencies across components
on interconnected buses, or signal and message propagation delays. These requirements are
incompatible with the use of several virtualization technologies, such as conventional type-1
hypervisors, due to overhead issues and the lack of support for real-time payloads.
However, recent developments such as the implementation of low-latency deterministic network
connectivity for converged Ethernet and the availability of real-time hypervisors made it possible to
virtualize components of the PLC architecture. The vPLC architecture hereby proposed (Figure 6)
takes advantage of these capabilities, by decoupling the PLC execution environment from I/O
modules – using an SDN-enabled Ethernet fabric to provide connectivity to the I/O subsystem. This
architecture departs from the SoftPLC concept, as proposed by products such as (Codesys) or
(ISaGRAF), by adopting an approach in line with (Intel 2013) and (IntervalZero 2011), with the added
benefit of a convergent fabric scenario with SDN capabilities.
Figure 6: The vPLC architecture
In the vPLC, the PLC I/O bus is replaced by high-speed networking capabilities, with SDN allowing for
the creation of flexible virtual channels on the I/O fabric, accommodating the connectivity flows
between the vPLC instances and the I/O modules, such as sensor interfaces or motion controllers,
providing traffic isolation. Moreover, such I/O modules can be built with reduced complexity, thanks to
recent progress in terms of Field-Programmable Gate Arrays (FPGA) and Application Specific
Connected I/O
modules Field
components
Virtualization
infrastructure
Convergent
communications
fabric
Physical host
HMI MS
Physical Host
vPLC HDBvPLCvPLC
FI
SDN
Controller
Physical Link
SDN flow
COTS Hypervisor
SDN vSwitch
RT
Hypervisor

Integrated Circuit (ASIC) technology. SDN reconfiguration is managed by means of an SDN
controller, via a High-Availability (HA) server (not depicted in the figure), which interacts with its
northbound interface. The HA server continuously monitors the SDN switch statistics and path
reachability, triggering reconfiguration procedures in case of performance degradation or failure.
This decentralized model shares similarities with remote or distributed I/O PLC topologies, with
networked I/O modules acting as extensions of the PLC rack. This goes in line with the Converged
Plantwide Ethernet (CWpE) (Didier 2011) architecture, or even critical avionics systems, which
replace legacy interconnects with Ethernet-based technologies, such as Avionics Full-Duplex
Switched Ethernet (AFDX) (Fuchs 2012).
Advances in cut-through switching, together with Remote Direct Memory Access techniques (RDMA),
particularly in converged Ethernet scenarios, have allowed for port-to-port latencies of the order of the
hundredths of nanoseconds in 10G Ethernet switch fabrics and application latencies in the order of
microseconds (Beck 2011). Additionally, resources such as Intel’s Data Plane Development Kit
(DPDK) (Zhang 2014) allow for the implementation of low latency, high-throughput packet processing
mechanisms that bypass kernels, bringing the network stack into user space and enabling adapters to
perform Direct Memory Access operations to application memory. This enables satisfying
requirements for single-digit microsecond jitter and restricted determinism, allowing for bare-metal
performance on commodity server hardware. On top of this, proposals such as the 802.1Qbv Time
Sensitive Networking (IEEE) standard provide compliance with real-time requirements in the
microsecond range on conventional Ethernet.
As for computing resources, there are two factors that must be considered. First, modern x86 or ARM
processors have become capable of replacing microcontrollers in standalone PLC applications (Kean
2010), due to improvements in terms of raw performance, low latency I/O mechanisms or the
availability of ISA extensions suitable for Digital Signal Processing tasks. Second, the availability of
real-time static partitioning hypervisors, such as Jailhouse (Siemens), Xtratum (Crespo 2010), X-Hyp
(X-HYP) or PikeOS (Baumann 2011) enables hosting RTOS guest VMs for real-time workloads.
Some hypervisors, such as Xtratum and PikeOS, even replicate the ARINC 653 (Fuchs 2012)
partitioning model for safety-critical avionics RTOS, with a Multiple Independent Levels of
Security/Safety (MILS) (Alves-Foss 2006) architecture.
The benefits of this approach are manifold. The price tag for entry-level PLCs is comparable to a
COTS server that can host several vPLC instances, being kept out of the factory floor or industrial
environment. Distributed I/O on converged Ethernet also provides cost-effective performance and
reliability benefits, as communications between different vPLC instances can take place across the
convergent fabric or even locally, if co-located on the same host, with SDN allowing for flexible
creation of communications channels, for differentiated requirements. Moreover, I/O modules – the
components with highest failure rate in PLCs – can be easily and quickly replaced, in case of failure.
Particularly, the potential advantages of the vPLC in terms of reliability, safety and security are
considerable, as it can take advantage of datacenter-like redundant power, computing and
communications resources. Other benefits are also envisioned, namely:
• Hypervisors allow for migration of virtualized ICS components, as well as instance cloning for
pre-deployment tests;
• PLC watchdogs and system-level debugging and tracing mechanisms can be implemented at
the hypervisor level, which is able to oversee and control the vPLC partition behavior;
• vPLCs benefit from partitioning isolation, with VMs being easy to restore in a fresh state in
case of tampering or other malicious activity;
• SDN-managed isolated I/O paths ease the implementation of flexible, on demand, protection
mechanisms at the I/O level (as shown in Section 3) also paving the way for the introduction
of NFV components at the ICS level.

Overall, these benefits suggest that virtualizing a PLC could be feasible even for a single instance per
device, using Industrial-grade Single Board Computers, instead of COTS servers.
5. Conclusion
This paper discussed the implications of the progressive introduction of virtualization technologies in
ICS, with a special focus on security and reliability aspects. The virtualization of both network and
computing virtualization was analyzed from an ICS-centric standpoint, covering recent developments
as well as proposing new use cases and approaches to improve network and systems security.
Starting with an overview of network virtualization technologies such as SDN and NFV and their
application within ICS and distributed IACS, the paper next addressed the issue of using hypervisor
technologies for real-time workloads. In this latter perspective, a virtual PLC (vPLC) architecture was
discussed, which transcends the simple virtualization of the PLC device, constituting an integrated
approach where the device merges with the infrastructure, in a seamless way. The vPLC takes
advantage of network and computing virtualization technologies to propose a converged approach for
plan-wide consolidation of the ICS infrastructure, with performance, cost and security benefits. This
proposal is presently under development by a team that includes the authors of this paper.
2 Acknowledgements
This work was partially funded by the ATENA H2020 Project (H2020-DS-2015-1 Project 700581).
References
Alves-Foss, J., Harrison, W., Oman, P., & Taylor, C. (2006). The MILS Architecture for High Assurance
Embedded Systems. International Journal of Embedded Systems, 2(3-4), 239–247.
Baumann, C., Bormer, T., Blasum, H., and Tverdyshev, S. (2011). Proving Memory Separation in a Microkernel
by Code Level Verification. In proc of the 14th IEEE International Symposium on Object/Component/Service-
Oriented Real-Time Distributed Computing (pp. 25–32).
Beck, M., and Kagan, M. (2011). Performance evaluation of the RDMA over Ethernet standard in enterprise data
center infrastructure. In proc of 3rd Workshop on Data Center - Convergent and Virtual Ethernet Switching.
Cahn, A., Hoyos, J., Hulse, M. and Keller, E. (2013) Software-Defined Energy Communication Networks: From
Substation Automation to Future Smart Grids. In proc of IEEE SmartGridComm 2013 Symposium - Smart Grid
Services and Management Models.
Chavez, A.R., Hamlet, J., Lee, E., Martin, M. and Stout, W. (2015) Network Randomization and Dynamic
Defense for Critical Infrastructure Systems. California, USA: Sandia National Laboratories.
Chiosi, M., et al. (2012). Network Functions Virtualization – An Introduction, Benefits, Enablers, Challenges &
Call for Action. Issue 1. ETSI White Paper. October 2012. Retrieved February 2016 from
http://portal.etsi.org/NFV/NFV_White_Paper.pdf.
Codesys GmbH. CODESYS Control RT: Real-time SoftPLC under Windows. Retrieved March 2016 from
https://www.codesys.com/products/codesys-runtime/control-rte.html.
Crespo, A., Ripoll, I., and Masmano, M. (2010). Partitioned Embedded Architecture Based on Hypervisor: The
XtratuM Approach. In Proc. of European Dependable Computing Conference (EDCC).
Cruz, T., Simões, P., Monteiro, E., Bastos, F., and Laranjeira, A. (2015). Cooperative security management for
broadband network environments. Security and Communication Networks, 8(18), 3953-3977.
Didier, P., and et al., F. M. (2011). Converged Plantwide Ethernet (CPwE) Design and Implementation Guide.
Dong X., Lin H., Tan R., Iyer R. and Kalbarczyk Z. (2015), Software-Defined Networking for Smart Grid
Resilience: Opportunities and Challenges, Proc. of 1st ACM Cyber-Physical System Security
Workshop (CPSS’15), Singapore, 2015.
ECI Telecom (2015) LightSEC NFV-based Cyber Security Solution for Utilities, Retrieved February 2016 from:
http://www.ecitele.com/media/1225/eci_lightsec_nfv_brochure-utilities.pdf
Ersue, M. (2013). ETSI NFV Management and Orchestration - An Overview”, Presentation at the IETF #88
Meeting, Vancouver, Canada, November 3-8, 2013. Retrieved February 2016 from: http://www.ietf.org/
proceedings/88/slides/slides-88-opsawg-6.pdf.
Fuchs, C. (2012). The Evolution of Avionics Networks From ARINC 429 to AFDX. In Proc. of Innovative Internet

Technologies and Mobile Communications and Aerospace Networks (Vol. 65, pp. 65–76).
Galup S. et al. (2009) ’An overview of IT service management, Communications of the ACM, 52(5), pp. 124-127,
2009, doi: 10.1145/1506409.1506439.
García-Valls, M., Cucinotta, T., and Lu, C. (2014). Challenges in real-time virtualization and predictable cloud
computing. Journal of Systems Architecture, 60(9), 726–740.
Genge, B., Haller, P., Beres, A., Sándor, H. and Kiss, I. (2016) Securing Cyber-Physical Systems. In Using
Software-Defined Networking to Mitigate Cyberattacks, pp. 305-329, 2016, Taylor & Francis Group.
IEEE, Time-Sensitive Networking Task Group. Retrieved February 2016 from: http://www.ieee802.org/1/
pages/tsn.html.
Igure, V.M.; Laughter, S.A. and Williams R.D. (2006) Security issues in SCADA networks, Computers; Security,
Volume 25, Issue 7, Pages 498-506, 2006.
Intel Corporation. (2013). Reducing Cost and Complexity with Industrial System Consolidation. Retrieved March
2016 from: http://www.intel.com/content/www/us/en/industrial-automation/reducing-cost-complexity-industrial
IntervalZero. (2010). A Soft-Control Architecture: Breakthrough in Hard Real-Time Design for complex Systems.
Retrieved from http://intervalzero.com/assets/wp_softControl.pdf
Irfan, N. and Mahmud, A. (2015) A Novel Secure SDN/LTE based Architecture for Smart Grid Security. In proc
of IEEE International Conference on Computer and Information Technology.
ISA-99.00.01 (2007) Security for Industrial Automation and Control Systems - Part 1: Terminology, Concepts,
and Models, American National Standard.
ISaGRAF. ISaGRAF Overview. Retrieved February 2016 from: http://www.isagraf.com.
Kang, D. et al., (2011) Proposal strategies of key management for data encryption in SCADA network of electric
power systems, Int. Journal of Electrical Power & Energy Sys., Vol. 33, Iss. 9, Nov. 2011.
Kean, L. (2010). Microcontroller to Intel Architecture Conversion: PLC Using Intel Atom Processor.
Kreutz, D., Ramos, F., Verissimo, P., Rothenberg, C., Azodolmolky, S. and Uhlig, S. (2014). Software Defined
Networking: A Comprehensive Survey. Proc. IEEE, 103(1), pp.14-76.
Krutz, R. L. (2006) Securing Scada Systems, USA: Wiley Publishing, Inc., 2006.
L. O’Murchu, N. Falliere (2011) W32.Stuxnet dossier, Symantec White Paper, February 2011.
Machii, W., Kato, I., Koike, M., Matta, M., Aoyama, T., Naruoka, H., Koshima I. and Hashimoto, Y. (2015)
Dynamic Zoning Based on Situational Activitie for ICS Security. In IEEE 978-1-4799-7862-5/15.
ONF (2012). OpenFlow Switch Specification, version 1.3.0 (Wire Protocol 0x04), Open Networking Foundation,
Proença, J., Cruz, T., Monteiro, E., and Simões, P. (2015). How to use Software-Defined Networking to Improve
Security–a Survey. In proc of the 14th European Conference on Cyber Warfare and Security 2015 (pp. 220).
RAD Data Communications Ltd. (2015) Megaplex-4 D-NFV Virtualization Module, Retrieved February 2016 from:
http://www.rad.com/Media/34173_D-NFV.pdf.
Siemens AG. Jailhouse Partitioning Hypervisor. Retrieved March 2016 from: https://github.com/siemens/jailhouse
Silva, E.G., Knob, L., Wickboldt, J., Gaspary, L., Granville, L. and Schaeffer-Filho, A. (2015) Capitalizing on SDN-
based SCADA systems: an anti-eavesdropping case-study. In IFIP 978-3-901882-76-0.
Triangle MicroWorks, Inc (2002) DNP3 Overview, Raleigh, North Carolina, Retrieved February 2016 from:
http://www.trianglemicroworks.com/documents/DNP3_Overview.pdf.
X-HYP Project. X-HYP Project. Retrieved February 2016 from: http://x-hyp.org.
Zhang, W., Wood, T., Ramakrishnan, K., and Hwang, J. (2014). Smartswitch: Blurring the line between network
infrastructure \and cloud applications. In Proc. of 6th USENIX Work. on Hot Topics in Cloud Computing.