Article

Optimal State Replication in Stateful Data Planes

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

In SDN stateful data planes, switches can execute algorithms to process traffic based on local states. This approach permits to offload decisions from the controller to the switches, thus reducing the latency when reacting to network events. We consider distributed network applications that process traffic at each switch based on local replicas of network-wide states. Replicating a state across multiple switches poses many challenges, because the number of state replicas and their placement affects both the data traffic distribution and the amount of synchronization traffic among the replicas. In this paper, we formulate the optimal placement problem for replicated states, taking into account the data traffic routing, to ensure that traffic flows are properly managed by network applications, and the synchronization traffic between replicas, to ensure state coherence. Due to the high complexity required to find the optimal solution, we also propose an approximated algorithm to scale to large network instances. We numerically show that this algorithm, despite its simplicity, well approximates the optimal solution. We also show the beneficial effects of state replication with respect to the singlereplica scenario, so far considered in the literature. Finally, we provide an asymptotic analysis to find the optimal number of replicas.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The optimal replication problem for multiple replicas has been defined and investigated in [16]. Given a network application and the corresponding states, the problem considers all the traffic flows that are affected by/affect such states and, based on a generic cost function, computes (i) the optimal number of replicas, (ii) their placement within the network and (iii) the corresponding optimal traffic routing. ...
... Given a network application and the corresponding states, the problem considers all the traffic flows that are affected by/affect such states and, based on a generic cost function, computes (i) the optimal number of replicas, (ii) their placement within the network and (iii) the corresponding optimal traffic routing. The work in [16] can be used as a building block for LOADER (i.e., the optimization engine), which provides the programming framework and the implementation for replicated states. ...
... The complexity is further increased when considering that a given state of an application may be accessed by different network applications such as in the case of two network applications reading a common counter. This inter and intra application dependency imposes a constraint on how the traffic must be routed across individual elements of the split application to ensure the correctness on the execution of the application [6,16]. ...
Article
Programmable data planes recently emerged as a prominent innovation in Software Defined Networking (SDN). They provide support for stateful per-packet/per-flow operations over hardware network switches specifically designed for network processing. Unlike early SDN solutions such as OpenFlow, modern stateful data planes permit to keep (and dynamically update) per-flow states local to each switch, thus dramatically improving reactiveness of network applications to different state changes. Still, in stateful data planes, the management of non-local states is assumed to be completely delegated to a centralized controller, thus requiring extra overhead to be accessed. Our LOADER proposal aims at contrasting the apparent dichotomy between local and non-local states. We do so by introducing a new possibility: permit to take localized (in-switch) decisions not only on local states but also on global replicated states, thus providing support for network-wide applications without incurring the drawbacks of classic approaches. To this purpose, (i) we provide high-level programming abstractions devised to define the states and the update logic of a generic network-wide application, and (ii) we detail the underlying low-level state management and replication mechanisms. We then show LOADER’s independence of the stateful data plane technology employed, by implementing it over two distinct stateful data planes (P4 switches and OPP – Open Packet Processor – switches), and by experimentally validating both implementations in an emulated testbed using a simple distributed Deny-of-Service (DoS) detection application.
... Network monitoring Collect statistics (e.g., number of packets per flow entry) [12,13] 2021 Flow size counter Report to the controller after the completion of collecting the size of flow in data plane [14] 2020 ...
Article
Full-text available
Software-defined networking (SDN) separates the control plane and the data plane, which provides network applications with global network topology and the flexibility to customize packet forwarding rules. SDN has a wide range of innovative applications in 5G, Internet of Things, and information center networks. However, the match-action programming model represented by OpenFlow/Protocol Oblivious Forwarding (POF) in SDN can only process limited types of data such as packets and metadata, making it hard to fulfill future network applications. In this paper, data type and data location are added in the matching fields and actions to make the match-action table (MAT) compatible with multiple types of data, hence improving the data plane’s programmability. Data type helps the MAT to perceive multiple types of data, allowing them to be processed by a single MAT. Data location allows MAT to be decoupled from data meaning, quickly locating specific data in the switch. Based on Intel’s Data Plane Development Kit (DPDK), we design and implement a pipeline that is compatible with multiple types of data processing. Protocol and data type oblivious match-action tables and atomic instructions are included in the pipeline. Experiments show that representing data with data type and data location makes the pipeline compatible with multiple types of data without sacrificing forwarding performance, fulfilling the needs of network applications to handle a variety of types of data while avoiding repeating hardware design.
... Additionally, data replication helps overcome long data transmission delays by keeping information close to the places where queries are generated [20,21]. Moreover, replications of many of the same datasets are dispersed during replication in the data network [22]. Different sites may have various functional features and capabilities [23], and coping of the same files on these sites is generally called a replica replacement problem [24]. ...
Article
Full-text available
The Peer to Peer-Cloud (P2P-Cloud) is a suitable alternative to distributed cloud-based or peer-to-peer (P2P)-based content on a large scale. The P2P-Cloud is used in many applications such as IPTV, Video-On-Demand, and so on. In the P2P-Cloud network, overload is a common problem during overcrowds. If a node receives many requests simultaneously, the node may not be able to respond quickly to user requests, and this access latency in P2P-Cloud networks is a major problem for their users. The replication method in P2P-Cloud environments reduces the time to access and uses network bandwidth by making multiple data copies in diverse locations. The replication improves access to the information and increases the reliability of the system. The data replication's main problem is identifying the best possible placement of replica data nodes based on user requests for data access time and an NP-hard optimization problem. This paper proposes a new replica replacement to improve average access time and replica cost using fuzzy logic and Ant Colony Optimization algorithm. Ants can find the shortest path to discover the optimal node to place the duplicate file with the least access time latency. The fuzzy module evaluates the historical information of each node to analyze the pheromone value per iteration. The fuzzy membership function is also used to determine each node's degree based on the four characteristics. The simulation results showed that the access time and replica cost are improved compared to other replica replacement algorithms.
Article
Modern 5G networks are capable of providing ultralow-latency and highly scalable network services by employing modern networking paradigms, such as software-defined networking (SDN) and network function virtualization. The latter enables performance-critical network applications to be run in a distributed fashion directly inside the infrastructure. Being distributed, those applications rely on sophisticated state replication algorithms to synchronize states among each other. Nevertheless, current implementations of such algorithms do not fully exploit the potential of the modern infrastructures, thus leading to suboptimal performance. In this article, we propose STARE, a novel state replication system tailored for 5G networks. At its core, STARE exploits stateful SDN to offload replication-related processes to the data plane, ultimately leading to reduced communication delays and processing overhead for virtual network functions. We provide a detailed description of the STARE architecture alongside a publicly-available P4-based implementation. Furthermore, our evaluation shows that STARE is capable of scaling to big networks while introducing low overhead in the network.
Conference Paper
Full-text available
Network conditions are dynamic; unfortunately, current approaches to configuring networks are not. Network operators need tools to express how a network's data-plane behavior should respond to a wide range of events and changing conditions, ranging from unexpected failures to shifting traffic patterns to planned maintenance. Yet, to update the network configuration today, operators typically rely on a combination of manual intervention and ad hoc scripts. In this paper, we present Kinetic, a domain specific language and network control system that enables operators to control their networks dynamically in a concise , intuitive way. Kinetic also automatically verifies the correctness of these control programs with respect to user-specified temporal properties. Our user study of Kinetic with several hundred network operators demonstrates that Kinetic is intuitive and usable, and our performance evaluation shows that realistic Kinetic programs scale well with the number of policies and the size of the network.
Article
Full-text available
Early programming languages for Software-Defined Networking (SDN) were built on top of the simple match-action paradigm offered by OpenFlow 1.0. However, emerging switches and middleboxes offer much more sophisticated support for persistent state in the data plane, without involving a central controller. In this paper, we introduce high-level programming abstractions and compiler technology that exploit these low-level mechanisms. Our SNAP language allows programmers to mix stateful programming with pure packet processing using global, persistent arrays indexed by packet-header fields. This allows SNAP programs to learn about the network environment, store per-flow information, and implement various stateful network functions. SNAP is high-level and modular, allowing flexible composition of independently-written programs on top of a "one-big-switch" abstraction. Our SNAP compiler analyzes programs to discover dependencies and race conditions, and offers a simple network transaction abstraction as a way to resolve the latter. A mixed integer-linear program optimizes both placement of state variables on switches and also routing traffic through them efficiently and in the right order. To compile programs efficiently, our compiler uses a novel variant of binary decision diagrams (BDDs), extended to incorporate stateful components. We have implemented a prototype compiler, applied it to a range of programs and topologies, and measured our techniques' scalability.
Article
Full-text available
Software-Defined Networking (SDN) is an emerging paradigm that promises to change the state of affairs of current networks, by breaking vertical integration, separating the network's control logic from the underlying routers and switches, promoting (logical) centralization of network control, and introducing the ability to program the network. The separation of concerns introduced between the definition of network policies, their implementation in switching hardware, and the forwarding of traffic, is key to the desired flexibility: by breaking the network control problem into tractable pieces, SDN makes it easier to create and introduce new abstractions in networking, simplifying network management and facilitating network evolution. Today, SDN is both a hot research topic and a concept gaining wide acceptance in industry, which justifies the comprehensive survey presented in this paper. We start by introducing the motivation for SDN, explain its main concepts and how it differs from traditional networking. Next, we present the key building blocks of an SDN infrastructure using a bottom-up, layered approach. We provide an in-depth analysis of the hardware infrastructure, southbound and northbounds APIs, network virtualization layers, network operating systems, network programming languages, and management applications. We also look at cross-layer problems such as debugging and troubleshooting. In an effort to anticipate the future evolution of this new paradigm, we discuss the main ongoing research efforts and challenges of SDN. In particular, we address the design of switches and control platforms -- with a focus on aspects such as resiliency, scalability, performance, security and dependability -- as well as new opportunities for carrier transport networks and cloud providers. Last but not least, we analyze the position of SDN as a key enabler of a software-defined environment.
Article
Full-text available
The CAP theorem asserts that any networked shared-data system can have only two of three desirable properties. However, by explicitly handling partitions, designers can optimize consistency and availability, thereby achieving some trade-off of all three. The featured Web extra is a podcast from Software Engineering Radio, in which the host interviews Dwight Merriman about the emerging NoSQL movement, the three types of nonrelational data stores, Brewer's CAP theorem, and much more.
Article
Full-text available
The Bayou architecture provides scalability, availability, extensibility, and adaptability features that address database storage needs of world-wide applications. In addition to discussing these features, this paper presents Bayou's mechanisms for permitting the replicas of a database to vary dynamically without global coordination. Key is the use of weak consistency replication among autonomous machines and strict adherence to the tenet that no operation should involve more than two machines. 1. Introduction Providing scalability, availability, extensibility and adaptability are some of the important challenges faced by designers of world-wide applications and the infrastructure upon which these applications are built. Successful applications running over the emerging global information superhighway must be scalable to support a potentially large, widely-distributed user community. The application's data must be available when any one of the many users needs it. The code base of th...
Article
Programmable data planes recently emerged as a prominent innovation in Software Defined Networking (SDN). They provide support for stateful per-packet/per-flow operations over hardware network switches specifically designed for network processing. Unlike early SDN solutions such as OpenFlow, modern stateful data planes permit to keep (and dynamically update) per-flow states local to each switch, thus dramatically improving reactiveness of network applications to different state changes. Still, in stateful data planes, the management of non-local states is assumed to be completely delegated to a centralized controller, thus requiring extra overhead to be accessed. Our LOADER proposal aims at contrasting the apparent dichotomy between local and non-local states. We do so by introducing a new possibility: permit to take localized (in-switch) decisions not only on local states but also on global replicated states, thus providing support for network-wide applications without incurring the drawbacks of classic approaches. To this purpose, (i) we provide high-level programming abstractions devised to define the states and the update logic of a generic network-wide application, and (ii) we detail the underlying low-level state management and replication mechanisms. We then show LOADER’s independence of the stateful data plane technology employed, by implementing it over two distinct stateful data planes (P4 switches and OPP – Open Packet Processor – switches), and by experimentally validating both implementations in an emulated testbed using a simple distributed Deny-of-Service (DoS) detection application.
Conference Paper
In Software Defined Networking (SDN) the control plane is physically separate from the forwarding plane. Control software programs the forwarding plane (e.g., switches and routers) using an open interface, such as OpenFlow. This paper aims to overcomes two limitations in current switching chips and the OpenFlow protocol: i) current hardware switches are quite rigid, allowing ``Match-Action'' processing on only a fixed set of fields, and ii) the OpenFlow specification only defines a limited repertoire of packet processing actions. We propose the RMT (reconfigurable match tables) model, a new RISC-inspired pipelined architecture for switching chips, and we identify the essential minimal set of action primitives to specify how headers are processed in hardware. RMT allows the forwarding plane to be changed in the field without modifying hardware. As in OpenFlow, the programmer can specify multiple match tables of arbitrary width and depth, subject only to an overall resource limit, with each table configurable for matching on arbitrary fields. However, RMT allows the programmer to modify all header fields much more comprehensively than in OpenFlow. Our paper describes the design of a 64 port by 10 Gb/s switch chip implementing the RMT model. Our concrete design demonstrates, contrary to concerns within the community, that flexible OpenFlow hardware switch implementations are feasible at almost no additional cost or power.
Conference Paper
With the rise of stateful programmable data planes, a lot of the network functions that used to be implemented in the controller or at the end-hosts are now moving to the data plane to benefit from line-rate processing. Unfortunately, stateful data planes also mean more complex network updates as not only flows, but also the associated states, must now be migrated consistently to guarantee correct network behaviors. The main challenge is that data-plane states are maintained at line rate, according to possibly runtime criteria, rendering controller-driven migration impossible. We present Swing State, a general state-management framework and runtime system supporting consistent state migration in stateful data planes. The key insight behind Swing State is to perform state migration entirely within the data plane by piggybacking state updates on live traffic. To minimize the overhead, Swing State only migrates the states that cannot be safely reconstructed at the destination switch. We implemented a prototype of Swing State for P4. Given a P4 program, Swing State performs static analysis to compute which states require consistent migration and automatically augments the program to enable the transfer of these states at runtime. Our preliminary results indicate that Swing State is practical in migrating data-plane states at line rate with small overhead.
Article
Software-defined networking (SDN) programs must simultaneously describe static forwarding behavior and dynamic updates in response to events. Event-driven updates are critical to get right, but difficult to implement correctly due to the high degree of concurrency in networks. Existing SDN platforms offer weak guarantees that can break application invariants, leading to problems such as dropped packets, degraded performance, security violations, etc. This paper introduces EVENT-DRIVEN CONSISTENT UPDATES that are guaranteed to preserve well-defined behaviors when transitioning between configurations in response to events. We propose NETWORK EVENT STRUCTURES (NESs) to model constraints on updates, such as which events can be enabled simultaneously and causal dependencies between events. We define an extension of the NetKAT language with mutable state, give semantics to stateful programs using NESs, and discuss provably-correct strategies for implementing NESs in SDNs. Finally, we evaluate our approach empirically, demonstrating that it gives well-defined consistency guarantees while avoiding expensive synchronization and packet buffering.
Article
Over the past 5-10 years, the rise of software-defined networking (SDN) has inspired a wide range of new systems, libraries, hypervisors and languages for programming, monitoring, and debugging network behavior. Oftentimes, these systems are disjoint—one language for programming and another for verification, and yet another for run-time monitoring and debugging. In this paper, we present a new, unified framework, called Temporal NetKAT, capable of facilitating all of these tasks at once. As its name suggests, Temporal NetKAT is the synthesis of two formal theories: past-time (finite trace) linear temporal logic and (network) Kleene Algebra with Tests. Temporal predicates allow programmers to write down concise properties of a packet’s path through the network and to make dynamic packet-forwarding, access control or debugging decisions on that basis. In addition to being useful for programming, the combined equational theory of LTL and NetKAT facilitates proofs of path-based correctness properties. Using new, general, proof techniques, we show that the equational semantics is sound with respect to the denotational semantics, and, for a class of programs we call network-wide programs, complete. We have also implemented a compiler for temporal NetKAT, evaluated its performance on a range of benchmarks, and studied the effectiveness of several optimizations.
Conference Paper
Early programming languages for software-defined networking (SDN) were built on top of the simple match-action paradigm offered by OpenFlow 1.0. However, emerging hardware and software switches offer much more sophisticated support for persistent state in the data plane, without involving a central controller. Nevertheless, managing stateful, distributed systems efficiently and correctly is known to be one of the most challenging programming problems. To simplify this new SDN problem, we introduce SNAP. SNAP offers a simpler "centralized" stateful programming model, by allowing programmers to develop programs on top of one big switch rather than many. These programs may contain reads and writes to global, persistent arrays, and as a result, programmers can implement a broad range of applications, from stateful firewalls to fine-grained traffic monitoring. The SNAP compiler relieves programmers of having to worry about how to distribute, place, and optimize access to these stateful arrays by doing it all for them. More specifically, the compiler discovers read/write dependencies between arrays and translates one-big-switch programs into an efficient internal representation based on a novel variant of binary decision diagrams. This internal representation is used to construct a mixed-integer linear program, which jointly optimizes the placement of state and the routing of traffic across the underlying physical topology. We have implemented a prototype compiler and applied it to about 20 SNAP programs over various topologies to demonstrate our techniques' scalability.
Article
In this paper, the expected distance between two uniformly distributed random points in a rectangle or in a rectangular parallelepiped is computed under three different metrics: the Manhattan metric, the Euclidean metric, and the Chebychev metric.
Conference Paper
Timely interaction between an SDN controller and switches is crucial to many SDN applications---e.g., fast rerouting during link failure and fine-grained traffic engineering in data centers. However, it is not well understood how the control plane in SDN switches impacts these applications. To this end, we conduct a comprehensive measurement study using four types of production SDN switches. Our measurements show that control actions, such as rule installation, have surprisingly high latency, due to both software implementation inefficiencies and fundamental traits of switch hardware.
Article
The emergence of programmable interfaces to network controllers offers network operators the flexibility to implement a variety of policies. We propose NetEgg, a programming framework that allows a network operator to specify the desired functionality using example behaviors. Our synthesis algorithm automatically infers the state that needs to be maintained to exhibit the desired behaviors along with the rules for processing network packets and updating the state. We report on an initial prototype of NetEgg. Our experiments evaluate the proposed framework based on the number of examples needed to specify a variety of policies considered in the literature, the computational requirements of the synthesis algorithm to translate these examples to programs , and the overhead introduced by the generated implementation for processing packets. Our results show that NetEgg can generate implementations that are consistent with the example behaviors, and have performance comparable to equivalent imperative implementations.
Article
In this survey we overview the definitions and methods for graph clustering, that is, finding sets of ''related'' vertices in graphs. We review the many definitions for what is a cluster in a graph and measures of cluster quality. Then we present global algorithms for producing a clustering for the entire vertex set of an input graph, after which we discuss the task of identifying a cluster for a specific seed vertex by local computation. Some ideas on the application areas of graph clustering algorithms are given. We also address the problematics of evaluating clusterings and benchmarking cluster algorithms.
Conference Paper
In Software Defined Networking (SDN) the control plane is physically separate from the forwarding plane. Control software programs the forwarding plane (e.g., switches and routers) using an open interface, such as OpenFlow. This paper aims to overcomes two limitations in current switching chips and the OpenFlow protocol: i) current hardware switches are quite rigid, allowing ``Match-Action'' processing on only a fixed set of fields, and ii) the OpenFlow specification only defines a limited repertoire of packet processing actions. We propose the RMT (reconfigurable match tables) model, a new RISC-inspired pipelined architecture for switching chips, and we identify the essential minimal set of action primitives to specify how headers are processed in hardware. RMT allows the forwarding plane to be changed in the field without modifying hardware. As in OpenFlow, the programmer can specify multiple match tables of arbitrary width and depth, subject only to an overall resource limit, with each table configurable for matching on arbitrary fields. However, RMT allows the programmer to modify all header fields much more comprehensively than in OpenFlow. Our paper describes the design of a 64 port by 10 Gb/s switch chip implementing the RMT model. Our concrete design demonstrates, contrary to concerns within the community, that flexible OpenFlow hardware switch implementations are feasible at almost no additional cost or power.
Article
In this article, we deconstruct scalability concerns in software-defined networking and argue that they are not unique to SDN. We explore the often voiced concerns in different settings, discuss scalability trade-offs in the SDN design space, and present some recent research on SDN scalability. Moreover, we enumerate overlooked yet important opportunities and challenges in scalability beyond the commonly used performance metrics.
Article
Network virtualization is recognized as an enabling technology for the future Internet. It aims to overcome the resistance of the current Internet to architectural change. Application of this technology relies on algorithms that can instantiate virtualized networks on a substrate infrastructure, optimizing the layout for service-relevant metrics. This class of algorithms is commonly known as "Virtual Network Embedding (VNE)" algorithms. This paper presents a survey of current research in the VNE area. Based upon a novel classification scheme for VNE algorithms a taxonomy of current approaches to the VNE problem is provided and opportunities for further research are discussed.
Conference Paper
Replicating data under Eventual Consistency (EC) allows any replica to accept updates without remote synchronisation. This ensures performance and scalability in large-scale distributed systems (e.g., clouds). However, published EC approaches are ad-hoc and error-prone. Under a formal Strong Eventual Consistency (SEC) model, we study sufficient conditions for convergence. A data type that satisfies these conditions is called a Conflict-free Replicated Data Type (CRDT). Replicas of any CRDT are guaranteed to converge in a self-stabilising manner, despite any number of failures. This paper formalises two popular approaches (state- and operation-based) and their relevant sufficient conditions. We study a number of useful CRDTs, such as sets with clean semantics, supporting both \add and \remove operations, and consider in depth the more complex Graph data type. CRDT types can be composed to develop large-scale distributed applications, and have interesting theoretical properties.
Article
Recent years have seen a surge of interest in gossip protocols, with proposals to apply them for purposes ranging from autonomic self-management, repair of inconsistencies, reliable multicast and distributed search. Yet the field of distributed computing is littered with technologies that had initial promise, but were ultimately rejected by the industry. Researchers who measure their work through its impact need to ask some tough, basic questions. What are the uses to which gossip is particularly well-matched, and what are its limitations? What alternatives are there to gossip-based solutions, and when would we be better-off using a non-gossip protocol? When, in effect, is gossip the technology of choice?
Article
It is demonstrated that the linear programming problem in d variables and n constraints can be solved in O(n) time when d is fixed. This bound follows from a multidimensional search technique which is applicable for quadratic programming as well. There is also developed an algorithm that is polynomial in both n and d provided d is bounded by a certain slowly growing function of n.
Article
Networks of coupled dynamical systems have been used to model biological oscillators, Josephson junction arrays, excitable media, neural networks, spatial games, genetic control networks and many other self-organizing systems. Ordinarily, the connection topology is assumed to be either completely regular or completely random. But many biological, technological and social networks lie somewhere between these two extremes. Here we explore simple models of networks that can be tuned through this middle ground: regular networks 'rewired' to introduce increasing amounts of disorder. We find that these systems can be highly clustered, like regular lattices, yet have small characteristic path lengths, like random graphs. We call them 'small-world' networks, by analogy with the small-world phenomenon (popularly known as six degrees of separation. The neural network of the worm Caenorhabditis elegans, the power grid of the western United States, and the collaboration graph of film actors are shown to be small-world networks. Models of dynamical systems with small-world coupling display enhanced signal-propagation speed, computational power, and synchronizability. In particular, infectious diseases spread more easily in small-world networks than in regular lattices.
Graph partitioning for network problems
  • K Ruddel
  • A Raith