Content uploaded by Barun Saha

Author content

All content in this area was uploaded by Barun Saha on Dec 10, 2020

Content may be subject to copyright.

Accepted Version (For Personal Use Only)

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing

this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

1

SAFAR: Simulated Annealing-based Flow

Allocation Rules for Industrial Networks

Barun Kumar Saha, Member, IEEE, Luca Haab, and Łukasz Podleski

Abstract—In this paper, we address the problem of optimal

ﬂow allocation in the context of industrial networks. The problem

is known to be computationally hard, which makes it difﬁcult

to solve for large networks. Moreover, industrial networks, such

as power grids, often have unique characteristics, for example,

integral ﬂows and delay-symmetric path requirements, which

further constrain the problem. To address these challenges, we

propose the Simulated Annealing-based Flow Allocation Rules

(SAFAR) scheme. At a high level, SAFAR identiﬁes the near-

optimal paths satisfying the bandwidth demands of a subset

of ﬂows by iteratively considering several conﬁgurations. The

optimality criteria herein corresponds to maximizing the number

of ﬂows for which bandwidth paths are obtained, as well as

improving the relative fairness in link utilization arising out of the

process. Subsequently, SAFAR assigns a fraction of bandwidth

(slice) to the remaining ﬂows. Finally, SAFAR generates appro-

priate forwarding rules for the switches. Results of performance

evaluation using real-life and synthetically-generated networks

reveal that SAFAR can allocate up to 99% ﬂows, which is an

improvement of up to 9.6% over a contemporary greedy scheme.

Moreover, SAFAR can improve the ﬂow allocation fairness

relatively by up to 37%. Finally, we also use a network of Open

vSwitch switches to verify the end-to-end functionality of SAFAR.

Index Terms—Flow allocation, industrial networks, simulated

annealing, optimization, fairness

I. INTRODUCTION

Communication networks today are increasingly becoming

softwarized and computation intensive [1]. For example, it

is expected that Intent-based Networks (IBNs) [2] would

automate the entire pipeline, starting from bidding and plan-

ning a network to commissioning and operating it. However,

regardless of whether a network has already been deployed

or is currently under planning, network engineers often need

to plan the installation of trafﬁc ﬂows pertaining to various

services running in the network, which gives rise to the optimal

network ﬂow allocation problem [3]–[7]. In other words, given

various network services, the objective is to determine the

paths that their respective packet ﬂows should take so that

certain performance parameters are optimized.

Although the aforementioned problem has been previously

studied, there is no one solution ﬁtting all scenarios due to

the different Quality of Service (QoS) [8]–[10] requirements

B. K. Saha is with Hitachi ABB Power Grids, Bangalore, India e-mail:

barun.kumarsaha@hitachi-powergrids.com.

L. Haab is with Hitachi ABB Power Grids, Bern, Switzerland e-mail:

luca.haab@hitachi-powergrids.com.

L. Podleski is with Hitachi ABB Power Grids, Bern, Switzerland e-mail:

lukasz.podleski@hitachi-powergrids.com.

of diverse networks. For example, industrial networks are

characterized by high determinism, high reliability, and low

latency. Although multipath communication [8], [11], where

packet stream of a ﬂow is split along more than one path, can

help in load balancing and improving throughput, they may

not be a very good choice for mission-critical applications.

In many industrial networking scenarios, a single path is

used by a given ﬂow in order to improve determinism vis-

`

a-vis latency and jitter. In particular, MPLS-TP, a widely used

transport network technology, has removed the provision of

multipaths available with the classical IP/MPLS in order to

eliminate non-deterministic behavior. Moreover, power grid

networks often require symmetrical upstream and downstream

latencies, such as for teleprotection services, without which

there could be grid blackouts and even disasters. Therefore,

in contrast to say, wireless [8], [11], [12] and data center

networks [13], [14], splitting a trafﬁc ﬂow along different paths

may not be feasible in power grid networks, in general. In

other words, industrial networks exhibit certain characteristics

that are widely different from other networks. It is noteworthy

that, in Industry 4.0, an efﬁcient communication network is a

key requirement to achieve seamless functioning. Therefore,

it is important that such networks are optimally managed.

Motivated by these aspects, we address the problem of optimal

ﬂow allocation in the context of industrial networks [15].

In particular, we consider a wired network with integral

(or unsplittable, i.e., passes entirely through a single path or

none at all) trafﬁc ﬂows having different QoS requirements.

For example, some ﬂows have high priority, whereas some

require delay-symmetric paths (detailed deﬁnitions are pro-

vided in a later Section). Our optimality criteria corresponds to

maximizing the number of ﬂows for which such paths can be

determined. In addition, we are also concerned with the overall

fairness of the ﬂow allocation process, where link bandwidth

utilizations are attempted to be kept at similar levels. This, for

example, can prevent scenarios where a given link is almost

not utilized, whereas another one is virtually saturated.

The optimal ﬂow allocation problems are known to be

computationally hard. In other words, it can take a long

time to generate an optimal solution, especially for power

grid networks having hundreds of nodes. This, in turn, also

demands high skill sets and efforts from network engineers,

which consequently, affects the cost.

One can solve such problems using greedy algorithms,

which exploit the solution space by always trying to ﬁnd a

better solution with lower cost, but which also run the risk

of getting trapped at local optima. Metaheuristics strategies,

on the other hand, use both exploitation and exploration. In

general, such algorithms always accept a solution with a lower

Accepted Version (For Personal Use Only)

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing

this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

2

Fig. 1: A high-level overview of SAFAR.

cost, but sometimes also accept one with a higher cost in

order to escape local optima. Simulated Annealing (SA) [16]–

[18] is a popular metaheuritics algorithm, which decreases

the probability of accepting a bad solution with time. We

use SA to design Simulated Annealing-based Flow Allocation

Rules (SAFAR), an end-to-end solution for optimal ﬂow

allocation in industrial networks. The use of SA is motivated

by its simplicity of design, long history of application, and

theoretical proof of convergence. Moreover, SA continues to

be a popular tool to solve hard optimization problems in

various engineering domains apart from networking [19]–[21].

As shown in Fig. 1, SAFAR essentially is a set of algo-

rithms. SAFAR begins by computing the near-optimal paths

for majority of the network ﬂows. However, depending upon

various factors, such as bandwidth availability, ﬂow demands,

and optimization cost, it may not be possible to allocate

dedicated bandwidth paths to all the ﬂows. To mitigate this

scenario, in the next step, SAFAR assigns a slice of band-

width to the otherwise unallocated ﬂows. Finally, SAFAR

generates the appropriate forwarding rules for the switches.

Subsequently, SAFAR, for example, can invoke the REST

API of a network management system to install the rules

at the switches. Except for the ﬁnal stage, SAFAR is fairly

generic and therefore, is applicable to different networks

irrespective of technology, for example, Software-deﬁned Net-

works (SDNs) and IP/MPLS/MPLS-TP networks. Moreover,

the switch rule installation phase can also be easily adapted for

a speciﬁc technology. Therefore, we envisage that SAFAR can

be used for planning a new network as well as for allocating

additional ﬂows to already operational industrial networks.

A. Contributions

The speciﬁc contributions of this work are as follows:

•Proposing SAFAR, an SA-based end-to-end near-optimal

integral bandwidth path allocation scheme for industrial

trafﬁc ﬂows with different requirements.

•Designing a proportional bandwidth slice allocation

scheme for the trafﬁc ﬂows deemed infeasible by the

optimal allocation process.

•Designing a solution to generate ready-to-install for-

warding rules for switches based on the aforementioned

bandwidth allocation techniques.

•Evaluating the proposed mechanisms via exhaustive sim-

ulations using real-life and synthetic networks of large

sizes, as well as by emulating a network of Open

vSwitch1(OVS) switches.

1https://www.openvswitch.org

B. Organization

The remainder of this paper is organized as follows: Section

II presents an overview of the contemporary and related

work. The ﬂow allocation problem and related aspects are

discussed in Section III. The optimal path ﬁnding algorithms

are presented in Section IV. Section V discusses the bandwidth

slicing technique and method for forwarding rules genera-

tion for switches. In Section VI, the experimental setup is

described, whereas the results are discussed in Section VII.

Finally, Section VIII concludes this work.

II. RE LATE D WORK

Efﬁcient ﬂow management is a recurrent task for any

operational network, which explains the volume of attention

[3]–[5], [7], [13], [22]–[25] that it continues to receive from

the contemporary researchers. This problem is also relevant to

other areas, such as agriculture [26] and transportation [27].

Trestian et al. [13] studied the challenges faced by long

trains of mice ﬂows (low volume, short lived, and many)

in data center networks. The authors proposed a dynamic

load balancing scheme, where the mice ﬂows are aggregated

and routed via weighted multipaths. Sehery and Clancy [23],

on the other hand, investigated the problem of optimal ﬂow

routing with Clos networks, and presented a heuristics-based

solution by transforming the representation of Clos networks.

The authors proposed to avoid hash collisions in Equal Cost

Multipath Routing by rerouting the elephant ﬂows (high vol-

ume, long lived, and few).

Layeghy et al. [22] obtained optimal ﬂow allocation in

SDNs using Constraint Programming (CP), where the objec-

tive function and constraints are modeled using a declarative

language. However, CP lack scalability, in general.

Thazin et al. [5] proposed a mechanism for end-to-end

dynamic bandwidth allocation in SDNs by considering the

QoS requirements of ﬂows and the utilization of links. To

mitigate congestion, an SDN controller reroutes high priority

ﬂows off the bottleneck links. Based on the allocations made

by controller, appropriate bandwidth are allocated at switch

queues. Takahashi et al. [4] addressed the problem of trafﬁc

engineering by segregating ﬂows into macroscopic groups and

by considering the change in trafﬁc volumes. The authors also

proposed prediction-based and load-balanced routing schemes

to ensure better network utilization. Saha et el. [28] observed

that the inherent heterogeneity of Internet of Things gives

rise to various kinds of QoS demands, such as delay- and/or

loss-sensitive communication. The authors designed a greedy

mechanism for routing ﬂows while taking the said QoS aspects

into account. Bera et al. [29] computed optimal forwarding

paths for a given set of ﬂows while trying to accommodate as

many ﬂows as possible, based on which the corresponding

forwarding rules were determined. The authors aimed at

minimizing the exactly-matching ﬂow rules installed at the

switches. Finally, on detecting congestion at the ﬂow tables,

some of the forwarding rules were redistributed. On a different

note, Koutensk´

y et al. [7] observed that, in contrast to the best-

effort service of the Internet, QoS goals can be better managed

using the Recursive InterNetwork Architecture (RINA), where

Accepted Version (For Personal Use Only)

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing

this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

3

an application can specify its QoS parameters, which are then

ensured by the ﬂow and resource allocators.

Wu et al. [11] addressed the problem of real-time trafﬁc

load distribution in order to improve the receivers’ goodput in

multipath networks. The proposed approach, by taking path

saturation and utility into consideration, allocates additional

bandwidth requirements of a ﬂow via its exiting path, or

identiﬁes another ﬂow that can free up resources. Wu et al. [8]

also investigated the quality-energy tradeoff for video trafﬁc

in mobile networks. In the proposed mechanism, low priority

video frames are dropped to obtain a revised ﬂow rate, while

maintaining a tolerable distortion level.

Saha et al. [25] investigated the problem of optimal ﬂow

allocation in industrial networks. The authors aimed at max-

imizing the number of integral ﬂows allocated as well the

ﬂow allocation fairness. The proposed Shortest Path-based

Allocation (SPA) scheme uses a greedy approach to allocate

all possible ﬂows. On the other hand, the SPA with SA (SPA-

SA) scheme uses metaheuristics to identify better solutions.

To synthesize, we ﬁnd that the problem of optimal ﬂow

allocation has been studied in different contexts, such as data

center networks. However, an investigation of the problem by

considering the unique characteristics of industrial networks,

as detailed in Section I, seems to be lacking, in general. For

example, multipath-based solutions seem to be unaffordable

when integral ﬂows are considered. Moreover, while the Clos

topology and RINA are promising, their adoption by industrial

networks, which consists of many ﬁeld devices using propri-

etary communication protocols, may be challenging. Although

Saha et al. [25] addressed the ﬂow allocation problem for

industrial networks, the authors discounted the consideration

of delay-symmetric path requirements, which is of importance

in power grid networks. Moreover, the prospect of improving

a solution by deleting an already allocated ﬂow—or real-

locating it to an alternative path—remained unconsidered.

Such deletion and reallocation operations, for example, are

very relevant to emergency scenarios, where certain ﬂows are

to be accommodated possibly at the cost of others. Some

other aspects, such as what to do with the unallocated ﬂows

and how logical allocations translate into physical ones, also

lacked in a detailed consideration. To address these challenges

and lacunae, in this paper, we take a detailed look into the

industrial network ﬂow allocation problem and present an end-

to-end solution for the same.

III. PROB LE M DESCRIPTION

Let G= (V, E )be a wired network, where Vand E,

respectively, are the sets of vertices (nodes) and edges (links).

We assume Gto be strongly connected, i.e., any node can

be reached from any other node. The links may or may not

bidirectional as long as strong connectivity is maintained. Let

Fbe the set of network trafﬁc ﬂows. Let FA⊆Fbe the set

of ﬂows already allocated in G. Then, F−FAdenotes the

set of unallocated ﬂows. Here, by “allocated” we mean that a

bandwidth path for the concerned ﬂow has been determined;

physical bandwidth allocation is not done at this stage.

Let |e|>0and |f|>0, respectively, be the bandwidth

capacity of any edge e∈Eand the bandwidth demand of

any ﬂow f∈F. Moreover, let f−1be the reverse (inverse)

of a ﬂow f, i.e., a new ﬂow with the source and target of

finterchanged. Given a ﬂow f, a bandwidth path Pfor f

is a simple path (i.e., a loop-free path) in Gthat begins and

ends, respectively, at the source and target of fsuch that the

residual bandwidth of all the edges in Pis greater than |f|.

We deﬁne a few additional terms in the following:

Deﬁnition 1. Link utilization Let Fx,E (f, e)be an indicator

function that takes the value 1only when a ﬂow f∈Fx⊆F

passes through an edge e∈E; otherwise, it takes the value

0. Moreover, let E,P(e, P )be another indicator function

that takes the value 1only when an edge e∈Eis part

of a path P∈ P, where Pis the set of all simple paths

between all pairs of nodes. Then, the (absolute) utilization

of any link e∈Efor a given set of ﬂows Fxis deﬁned as

uFx(e) = P

f∈FxP

Pf∈P

Fx,E (f, e)E ,P(e, Pf)|f|, where Pfis

the bandwidth path used by ﬂow f.

Deﬁnition 2. Relative path utilization Let Pbe a band-

width path. The relative utilization of path Pis a vector

of fractional utilization of all its links, and is deﬁned as

vFx(P) = [1

|e|uFx(e)]e∈P.

Deﬁnition 3. Delay-symmetric path Two bandwidth paths P1

and P2,P16=P2, are said to be delay-symmetric if both the

paths have equal delay between their sources and destinations.

Note that, for any edge e∈E,uFA(e)denotes its current

utilization. On the other hand, uFA∪{f}(e)denotes the pro-

jected utilization of eby considering the hypothetical addition

of another ﬂow fto FA;f6∈ FA. These formulations will be

used later in this work.

We consider the problem of optimal bandwidth path allo-

cation for integral ﬂows, which is of relevance to industrial

networks. Our deﬁnition of “optimality” involves the following

two objectives: (1) maximizing the number of ﬂows that are

allocated (i.e., the set FA) and (2) improving the average

fairness in ﬂow allocation in the network (say, J). The latter

objective requires that all links are uniformly utilized to the

extent possible. This requirement can help to obtain better

distribution of trafﬁc load across the network, on average,

which can prevent the links from getting saturated soon.

Let Q1, Q2, . . . , Qkbe the different QoS attributes that

deﬁne a ﬂow. Let |Qi|be the number of (discrete) values

that the QoS attribute Qican take. For example, the attribute

ﬂow priority may have two values, high and low. Accord-

ingly, there are |Q1|×|Q2|×···×|Qk|distinct QoS trafﬁc

classes, C0, C1, . . . , C|Q1|×|Q2|×···×|Qk|−1, and each ﬂow in

Fis mapped into exactly one such class2;S

i

Ci=F;

Ci∩Cj=∅,∀i, j, i 6=j. Based on how these attributes are

deﬁned, C0, for example, may be the highest priority QoS

class; a partial ordering Ci<Cj,i6=j, may be induced

2Having QoS classes encompass the individual metrics is useful in practice.

For example, certain bit patterns can be set in IP headers to indicate what

kind of QoS the packets need. A relative ranking of the classes is obtained

based on the application requirements. For example, MPLS assigns highest

priority to voice, followed by premium services and best-effort services. A

detailed discussion on this topic, however, is scoped out of the work.

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

4

among the other classes. We also use the notation f∈Cito

indicate that a ﬂow f∈Fbelongs to the QoS class Ci.

In this work, we consider the following two QoS attributes:

•High priority: Some ﬂows in Fhave high priority, for

example, voice data, whereas the others do not.

•Delay-symmetric path: Some upstream-downstream

ﬂow pairs (i.e., ﬂows from sto tand tto s) require

that their paths be delay-symmetric (see Deﬁnition 3),

whereas the other ﬂows do not.

Accordingly, we deﬁne the following classes: C0={high

priority, delay-symmetric path},C1={high priority, ¬delay-

symmetric path},C2={¬high priority, delay-symmetric

path}, and C3={¬high priority, ¬delay-symmetric path}.

The ﬂows belonging to class C0must be allocated, whereas

allocation for the other ﬂows is not mandatory.

We also constrain the utilization of each link by an upper

limit, 0< θ < 1. This helps us to keep a fraction of link

bandwidth free, which can be used to mitigate trafﬁc bursts

and to accommodate future ﬂows. In addition, we use Jain’s

Fairness Index [30] to measure the overall fairness in ﬂow

allocation. We deﬁne Jas follows:

J=

P

e∈E

uFA(e)2

|E|P

e∈E

uFA(e)2(1)

It may be noted that θand Jare complementary in nature—

while θupper bounds the links’ usage, Jtries to keep such

usages at similar levels. Accordingly, the cost function is

deﬁned as follows:

c(G, F ) = α(1 −J) + (1 −α)1−|FA|

|F|(2)

where 0< α < 1is a control parameter. Finally, the optimal

network ﬂow allocation problem is deﬁned as follows:

minimize

Fc(G, F )(3a)

subject to 0 ≤uFA(e)≤θ|e|,∀e∈E(3b)

X

P∈P

FA,P(f, P )=1,∀f∈FA(3c)

C0⊂FA(3d)

where the indicator function FA,P(f , P )is 1if a ﬂow f∈FA

passes through a path P∈ P;0otherwise. The constraint (3b)

indicates that the total volume of trafﬁc passing through any

edge must not exceed θtimes its capacity. In other words,

the total volume should be comfortably lower than an edge’s

maximum capacity. The constraint (3c) indicates that any

allocated ﬂow must pass through exactly one path between its

source and destination. Finally, the constraint (3d) indicates

that all C0ﬂows are to be allocated.

Let us consider that all the ﬂows in C0are allocated so

that Fis reduced to say, F0, and G= (V, E)is reduced

to say, G0= (V, E0). Then, with the constraint (3d) gone,

the aforementioned problem essentially translates into a mul-

ticommodity network ﬂow (MCNF) problem [23], [31]–[33].

MCNFs are computationally hard problems. The integral

MCNF problem—pertaining to the integral ﬂow allocation

TABLE I: Summary of notations

Symbol Deﬁnition

V,ESet of nodes and links, respectively

F,FASet of input and allocated ﬂows, respectively

|e|Bandwidth capacity of an edge e

|f|Bandwidth demand of a ﬂow f

f−1Reverse (inverse) ﬂow of f, i.e., a ﬂow the from target

of fto the source of f

PSet of all simple paths between all pairs of nodes

P1,P2Set of shortest and alternative paths, respectively

uFA(e)Bandwidth utilization of any edge ebased on the

allocated ﬂows set FA

vFA(P)Vector of relative utilizations of all edges in any path

Pbased on the allocated ﬂows set FA

FA,E (f, e),

E,P(e, P ),

FA,P(f, P )

Indicates whether or not any ﬂow fuses a link e, any

edge eis in a path P, and any ﬂow fpasses through

a path P, respectively

Q1, Q2,... QoS attributes

C0, C1,... QoS classes

θLink utilization threshold

scenario—is NP-complete and difﬁcult to solve even for small

networks [33]. Moreover, commercial solvers may not be

able to ﬁnd feasible solutions for MCNFs when the problem

size increases [34]. Consequently, heuristics are used to solve

such problems [23], [34]. Motivated by these, we design a

metaheuristics solution, SAFAR, to solve the problem. Table

I summarizes Some of the key notations used in this work.

Before closing this Section, we note that wired links are

signiﬁcantly less prone to packet loss than wireless links,

where interference, fading, and other aspects lead to packet

loss. Although certain external and internal factors, such as

ﬁber cut and congestion, also affect wired networks, they are

not necessarily the characteristics of the links. Consequently,

we assume that the wired paths are lossless. Moreover, not

all ﬂows need to be loss-free; certain services, such as audio

streaming, can be subjected to packet loss. SAFAR allocates

the C0ﬂows before others, which ensures that bandwidth

demand of the most important services are met. In addition,

SAFAR also leaves a fraction of links’ bandwidths unused,

which allows to accommodate trafﬁc bursts to some extent.

We also assume that the switches have sufﬁcient capacity

available to store all the necessary forwarding rules. This

assumption is supported by several contemporary works aimed

at mitigating the limitations of ﬂow table sizes, for example, by

redistributing the ﬂow rules when a table gets congested [29],

aggregating and compressing ﬂow rules [35], and using virtual

ﬂow tables [36]. Accordingly, one or more such strategies may

be periodically used to manage the storage space for ﬂow rules.

IV. SIM UL ATED ANNEALING-BAS ED FL OW ALL OC ATIO N

In this Section, we describe the SA-based SAFAR in details.

The mechanisms for forwarding rule generation for switches

would be discussed in a subsequent Section.

A. Overview of Simulated Annealing

Annealing involves heating metals to a high temperature

and then cooling down very slowly in order to attain a perfect

crystalline structure devoid of impurities. SA uses a similar

strategy to ﬁnd solutions of hard optimization problems.

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

5

SA begins by creating a feasible initial solution (S0). Subse-

quently, SA iterates over several steps to identify an optimal or

near-optimal solution. Let c1and c2, respectively, be the costs

of the previous and current solutions, so that ∆ = c2−c1

is the difference in costs. The current solution is deﬁnitely

selected when it incurs a lower cost, i.e., when ∆<0.

Otherwise, i.e., when ∆≥0, the current solution is selected

with probability exp(−kB∆

T), where Tis the contemporary

system “temperature” (a large positive number) and kBis the

Boltzmann constant. At the end of an iteration, the value of T

is decreased by a small factor following an appropriate cooling

schedule, such as linear and exponential. SA terminates when

Tbecomes zero. At that point, the best solution selected so

far is considered as the optimal or near-optimal result.

B. Initial Solution

A (feasible) solution S⊆Fis a collection of ﬂows such

that the bandwidth demand of each ﬂow in Sis satisﬁed and

utilization of no link in the network exceeds θbecause of

S(see (3b)). An initial solution, S0, deﬁnes a (sub-optimal)

starting point of SAFAR, which is further optimized by SA.

As shown in Algorithm 1, SAFAR creates an initial solution

by greedily allocating as many ﬂows as possible by iterating

over Fin a speciﬁc order. In particular, Algorithm 1 tries to

allocate the ﬂows from C0at ﬁrst (line numbers 1through

6). Since C0represents the highest priority QoS class, this

ensures that the ﬂows from this class get their bandwidth

demands satisﬁed before others. Subsequently, the classes C1,

C2, and C3are iterated over in that order. Note that Algorithm

1 assumes that the links are bidirectional, which is why line

numbers 4and 13 are valid. In case of unidirectional links—as

we consider in our evaluations—this can be easily modiﬁed

by considering a different physical path for f−1.

C. Neighbor Selection

Starting with S0, SAFAR iteratively picks up a random

“neighbor” and evaluates the prospect of moving to that

neighboring state. Algorithm 2 presents the detailed steps for

solution (neighbor) creation in SAFAR. In particular, given a

solution S, a random neighbor of Sis generated by 1) deleting

the ﬂow indicated by S−from S, 2) adding the new ﬂow(s)

indicated by S+to S, and 3) relocating the existing ﬂow(s)

indicated by S∼in Svia alternative paths. These operations

are done in such a way so that the usage threshold θis not

breached for any link in G.

Algorithm 2 shows that we delete only an already allocated

ﬂow from C3and that too with a small probability, pdel (line

numbers 2and 3). This ensures that any ﬂow from any higher

QoS class, if allocated, is never removed. Moreover, deleting

ﬂows at every iteration—which are many in SA—may lead to

unnecessary computations. Subsequently, we add—provided

feasible paths are available—up to φunallocated ﬂows from

each QoS class (in order or decreasing importance) (line

numbers 4through 10) and relocate—provided an alternative

feasible path is available—up to φalready allocated ﬂows from

all the classes (line numbers 11 through 17).

Algorithm 1: Initial solution generation

Input:

•F: Set of ﬂows to allocate

•P1: Set of all-pair shortest paths

•θ: Link utilization threshold

Output: FA: Set of allocated ﬂows

1foreach f∈C0do

2P← P1[f]// Primary path for flow f

3emax = arg max vFA∪{f ,f−1}(P)// The edge in

Pwith projected maximum relative

utilization considering fand f−1

4if uFA∪{f,f −1}(emax)< θ|emax |then

5Allocate falong P

6Allocate f−1along Pwith links reversed

7foreach f∈C1do

8emax = arg max vFA∪{f}(P1[f])

9if uFA∪{f}(emax)< θ|emax|then

10 Allocate falong P1[f]

11 foreach f∈C2do

12 emax = arg max vFA∪{f ,f−1}(P1[f])

13 if uFA∪{f,f −1}(emax)< θ|emax |then

14 Allocate falong P1[f]

15 Allocate f−1along P1[f]with links reversed

16 foreach f∈C3do

17 emax = arg max vFA∪{f}(P1[f])

18 if uFA∪{f}(emax)< θ|emax|then

19 Allocate falong P1[f]

Before SAFAR begins, we use Dijkstra’s shortest path algo-

rithm to compute P1, the set of all-pair shortest paths, based on

latency of the paths. In addition, we use Yen’s algorithm [28],

[37] to compute P2, the set of second-shortest paths among

all node pairs. However, in order to avoid high computational

times for large networks, we consider an alternative path by

replacing an edge of the shortest path with two other edges.

Since such replacements are not always feasible, alternative

paths may not available for some source-target node pairs. P1

and P2are computed only once and need not be re-computed

again, unless there is any change in the network topology.

D. Optimal Flow Allocation

Having described the key components, we present the

complete steps of optimal ﬂow allocation in Algorithm 3. The

Algorithm attempts to minimize the cost function deﬁned in

(2). Algorithm 3 begins by creating a greedy initial solution

(line number 1) and computes its cost (line number 3).

Subsequently, Algorithm 3 repeatedly generates a new solution

(line numbers 6) and computes its cost (line numbers 7). If the

current solution has a lower cost than the previous solution,

i.e., the cost difference, ∆, is negative, the current solution is

deﬁnitely accepted (line number 10); otherwise, it is accepted

with probability exp(−kB∆

T)(line number 12). By “accepting,”

we mean that the current allocations for the concerned ﬂows

are recorded, which however, could change at a later point. In

line number 2, we estimate the value of Boltzmann constant,

kB, using the relation kB=Tln 0.8

¯

∆E[38], where 0.8is the

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

6

Algorithm 2: Solution generation for SA

Input:

•F: Set of ﬂows to allocate

•FA: Set of already allocated ﬂows

•P1: Set of all-pair primary shortest paths

•P2: Set of all-pair secondary shortest paths

•θ: Link utilization threshold

•φ: Maximum number of ﬂows in a solution

•pdel: Flow deletion probability

Output: S+, S−, S∼: Set of ﬂows to add, delete, and

relocate, respectively

1S+, S−, S∼← ∅,∅,∅

2f←Randomly selected ﬂow from C3∩FA

3Add fto S−with probability pdel

4foreach C∈[C0, C1, C2, C3]do

5repeat φtimes

6f←Randomly selected ﬂow from C∩(F−FA)

7P← P1[f]

8emax = arg max vFA∪{f}∪S−(P)

9if uFA∪{f}∪S−(emax)< θ|emax|then

10 Add (f, p)to S+

11 foreach C∈[C0, C1, C2, C3]do

12 repeat φtimes

13 f←Randomly selected ﬂow from C∩FA

14 P, P2← P1[f],P2[f]

15 emax = arg max vFA∪{f}∪S−∪S+(P2)

16 if P26=∅and uFA∪{f}∪S−∪S+(emax)< θ|emax |

then

17 Add (f, P2)to S∼

Algorithm 3: Optimal ﬂow allocation using SAFAR

Input:

•F: Set of ﬂows to allocate

•T: (Initial) temperature

•γ: Cooling coefﬁcient

•η: Number of iterations

Output: FA: Set of allocated ﬂows

1FA←Initial solution using Algorithm 1

2Estimate the value of kB

3c0←Cost of FAusing (2)

4while T > 0do

5repeat ηtimes

6S←New solution using Algorithm 2

7c←Cost of Susing (2)

8∆←c−c0

9if ∆<0then

10 A←1

11 else

12 A←exp( −kB∆

T)

13 r←Uniform random number between 0and 1

14 if r < A then

15 c0←c

16 Update FAby adding, removing, and relocating

ﬂows indicated by S

17 T← bγT c

initial value of acceptance probability. The value of ¯

∆Eis

estimated by perturbing the initial solution 1000 times [38].

Algorithm 3 executes a loop until Tis zero. Line number

17 presents the cooling schedule, where Tis scaled down by

a factor 0< γ < 1, ensures that Ttakes only integer values,

which helps to reduce the number of iterations. Moreover, at

any given temperature, several iterations are performed (line

number 5). Overall, the parameters T,η, and γpresent a

trade-off between the speed of execution and the degree of

optimality. When Algorithm 3 concludes, the set FAgives the

set of ﬂows for which optimal bandwidth paths are determined.

E. Analysis

We now take a look at the time complexities of these

Algorithms. We recall that the set of paths, P1and P2,

are already computed before SAFAR is executed. Let ¯

Pbe

the average length of any path. For any given network, the

parameter ¯

Pdepends on shortest path computation and ﬂow

allocation techniques, but usually is a small number in the

order of 10. For simplicity, we treat it as a constant.

The outer loops in Algorithm 1, considered together, run for

|F|times. Line number 3, as well as 5and 6each, requires

¯

Piterations. Consequently, the time complexity of Algorithm

1 becomes O(¯

P|F|)or O(|F|).

The outer loop in Algorithm 2 runs for a constant number

of times for a given network, whereas the inner loop repeats

φtimes. Line numbers 8and 15 each iterates for κ1¯

Ptimes,

where κ1is a positive constant. Then, the time complexity

of Algorithm 2 becomes O(κ1¯

P)or O(1). This makes sense,

because given a solution, we only alter it with a few ﬂows.

In Algorithm 3, the two loops (line numbers 4and 5)

considered together iterate for about ηlog 1

γT0times, where

T0is the initial temperature. Inside the inner loop, cost

computation (line number 7) requires |E|iterations, whereas

updating FA(line number 16) requires κ2¯

Piterations, where

κ2is a positive constant. Note that line number 1invokes

Algorithm 1. Consequently, the time complexity of Algorithm

3 becomes O(max{η|E|log 1

γT0,|F|}). Typically, we have

|E|<|F|. However, for complete graphs, |E|=|F|=|V|

2.

Moreover, for asymptotically large networks, ηlog 1

γT0

|E|. Therefore, the asymptotic time complexity of the optimal

ﬂow allocation scheme becomes O(|E|) = O(|F|).

V. BANDWIDTH SLI CE S AN D FLOW RU LE S GEN ER ATION

We now discuss the last two stages of SAFAR (see Fig. 1).

A. Proportional Slice Allocation

At the end of Algorithm 3, some ﬂows may remain “un-

allocated,” i.e., |FA| 6=|F|. Algorithm 4 assigns bandwidth

slices to those unallocated ﬂows. The key idea here is to ﬁnd

the minimum volume of data that a ﬂow can transfer along

all the links of its shortest path. Such minimum volume is

proportional to a ﬂow’s bandwidth demand.

Since several ﬂows can move along a particular link, Algo-

rithm 4 begins by computing the trafﬁc load offered by all such

ﬂows to all the links in a network (line numbers 1through 6).

For any edge e∈E, if `[e]is less than the residual capacity

of e, then the bandwidth demands of all ﬂows passing through

that link are met. However, if `[e]exceeds the link capacity,

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

7

Algorithm 4: Bandwidth slice allocation

Input:

•F: Set of ﬂows to allocate

•FA: Set of already allocated ﬂows

•P1: Set of all-pair shortest paths

•θ: Link utilization threshold

Output: ς: Bandwidth slices for all ﬂows

// Load offered to the edges

1foreach e∈Edo

2`[e]←0

3foreach f∈F−FAdo

4if e∈ P1[f]then

5`[e]←`[e] + |f|

// Proportional load of each flow

6foreach e∈Edo

7foreach f∈F−FAdo

8℘[e][f]←min{|f|×θ×|e|

`[e],|f|}

// Min. among all proportions

9foreach f∈F−FAdo

10 ς[f]←min

e∈P1[f]℘[e][f]

11 foreach f∈FAdo

12 ς[f]← |f|

then each ﬂow passing through that link gets a bandwidth slice

proportional to the ﬂow’s demand (line numbers 7through 8).

At the end of this step, a given ﬂow may be assigned different

proportions for the different links of the path it passes through.

The minimum of these constitutes the bandwidth slice assigned

to the concerned ﬂow (line number 10 through 12).

Finally, in line number 8, we again consider the link usage

threshold, θ. This helps to keep a fraction of the residual

bandwidths free for other uses, for example, future ﬂow

allocations and accommodation of trafﬁc bursts.

To illustrate, let us consider a linear topology of three

switches say, A,B, and C, where the links AB and BC each

has a bandwidth capacity of 10 unit. Let fA→Band fA→Cbe

two ﬂows, where fA→Bhas a demand of 12 unit from Ato

Band fA→Chas a demand of 14 unit from Ato C. Clearly,

the demands of neither of these ﬂows can be fully met. To

compute the bandwidth slices, we note that the two ﬂows offer

a load of 12+14 = 26 unit on the edge AB, whereas the edge

BC has an offered load of 14 unit. Assuming θ= 0.75, the

proportional load of fA→Bfor AB is 0.75×12×10

26 ≈3.46 unit;

of fA→Cfor AB is 0.75×14×10

26 ≈4.04 unit; of fA→Cfor BC

is 0.75×14×10

14 = 7.5unit. Consequently, fA→Bis allocated

3.46 unit of bandwidth, whereas fA→Cis allocated 4.04 unit.

B. Forwarding Rules

Algorithm 5 illustrates the steps of generating forwarding

rules for switches based on bandwidth slicing [39]. This Algo-

rithm is at fairly high level, and assumes that forwarding rules

are applied to the egress ports of switches, which, for example,

is supported by OpenFlow. Similar rules for the ingress ports

can be generated as well with minor modiﬁcations.

Algorithm 5 begins with representing the forwarding paths

in terms of ports rather than links (line numbers 4through 11).

Algorithm 5: Flow rules installation

Input:

•F: Set of ﬂows

•P: Paths allocated to the ﬂows

•ς: Bandwidth slices

•Φ: Filter to identify ﬂows

Output: Flow rules installed in all switches

1foreach s∈Vdo

2foreach egress port xof sdo

3Π[x]← ∅

// Identify port and queue requirements

4foreach f∈Fdo

5X←Sequence of egress ports for the links in P[f]

6foreach x∈Xdo

7q←New queue identiﬁer

8rmax ←ς[f]

9rmin ←rmax/2

10 Add (q, rmin , rmax,Φ[f]) to Π[x]

// Prepare and create queues for each port

11 foreach π∈Πdo

12 e←Link attached to port π

13 Preapare HTB QoS with |e|as the max. rate

14 foreach (q, rmin, rmax , φ)∈Π[x]do

15 Prepare new queue (q, rmin, rmax , φ)

16 Create the QoS queues

// Create flow actions

17 foreach s∈Vdo

18 foreach egress port xof sdo

19 Add ﬂow action entry to susing queue, rates, and

ﬁlter from Π[x]

Subsequently, for a given ﬂow exiting a particular port, the

Algorithm evaluates the minimum and maximum bit rates to

be assigned to the corresponding QoS queue. For this purpose,

we assume that each ﬂow can be uniquely identiﬁed with

a set of ﬁlter or matching attributes (Φ), such as network

source, destination, and input port. These attributes are used

to create forwarding rules, which are matched by switches

to determine packet actions. The subsequent lines generate

the conﬁgurations necessary to actually create the queues.

Without loss of generality, we assume that the Hierarchical

Token Bucket (HTB) queue discipline is used.

Finally, line numbers 18 through 20 install the forwarding

rules in the switches. Following OpenFlow’s philosophy, we

depict queue creation and ﬂow rule installation as separate

tasks. Depending upon the network and NMS, these two tasks

can be combined into a single operation.

Listing 1 shows some sample (partial) output of Algorithm

5 for an example network. The ﬁrst command creates rate-

limited queues at switch s4port s4−eth5, whereas the second

command installs an OpenFlow ﬂow rule at the switch.

sudo ovs-vsctl set port s4-eth5 qos=@newqos -- \

--id=@newqos create QoS type=linux-htb \

other-config:max-rate=100000 \

queues:1002=@1q \

queues:1003=@2q \

queues:1005=@3q -- \

--id=@1q create queue other-config:min-rate=10000 other-config:max-

rate=20000 -- \

--id=@2q create queue other-config:min-rate=12500 other-config:max-

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

8

rate=25000 -- \

--id=@3q create queue other-config:min-rate=31875 other-config:max-

rate=63750

sudo ovs-ofctl -O Openflow13 add-flow s4 priority=101,in_port=1,

eth_src=00:00:00:00:00:01,eth_dst=00:00:00:00:00:04,

idle_timeout=0,actions=set_queue:1005,output:5

Listing 1: Sample output commands

In Algorithm 4, line numbers 1and 3, respectively, execute

for |E|and |F−FA|times, which results in a time complexity

of O(|E||F−FA|)≈O(|V|2|F|). In practice, |F−FA||F|,

which implies that the Algorithm has a fast running time. In

Algorithm 5, the nested loops in line numbers 1–3run for |V|δ

times, where δis the average number of active ports (degree)

any switch has. On the other hand, the loops in line numbers

4–10 run for about |F|¯

Ptimes. Since |F||V|, the average

time complexity of Algorithm 5 is O(|F|¯

P) = O(|F|).

VI. PERFORMANCE EVALUATIO N

In this Section, we describe our experimental setup and

metrics used to evaluate the performance of SAFAR.

A. Experimental Setup

We implemented SAFAR using Python 2.7 in a 64-bit Linux

system equipped with Intel(R) Core(TM) i5-4570S CPU @

2.90 GHz quad-core processor and 8 GB RAM. We considered

four real-life network topologies [40]—labeled as N79, N87,

N138, and N315, respectively, in Table II—to evaluate the

performance of SAFAR. Various aspects of these networks and

ﬂow demands (appropriate units assumed) are summarized in

the Table. In addition, to investigate the scalability aspects

of SAFAR, we also considered a synthetic network topology,

N500, which is a random graph with 500 vertices and link

existence probability of 0.35. The link bandwidths and ﬂow

demands for N500 were generated uniformly at random.

We also simulated other similar schemes to compare the per-

formance of SAFAR with. In particular, we considered SPA-T

[25], a greedy scheme for ﬂow allocation, as a benchmark. We

also considered the initial solution of SA (labeled as “Initial”),

which helped to identify the extent of improvement that SA

provided over a greedy technique. In addition, we implemented

the k-OPT optimal ﬂow allocation scheme, which performs an

exhaustive search by considering up to ksimple paths between

any pair of nodes. We compared the performance of SAFAR

with 2-OPT and 4-OPT to gain better insights.

Table III shows the simulation parameters. Note that the

relative sizes of C1and C2are shown in the Table. Accord-

ingly, |C0|=|C1||C2|and C3= 1 − |C0|−|C1|−|C2|. In

addition, we assumed pdel = 0.00005. Each experiment was

repeated 10 times, based on which the mean value and the

corresponding 95% conﬁdence interval were computed.

In order to verify the correctness of the switching rules

generated by SAFAR and the end-to-end functionality of

SAFAR, we also emulated a network of OVS switches using

Mininet3in a virtual machine with 4GB main memory. Fig. 2

shows the corresponding experimental network consisting of

3http://mininet.org/

TABLE II: Network topologies and ﬂow demands

N79 N87 N138 N315 N500

Nodes 79 87 138 315 500

Links 294 322 744 1944 144206

Avg. degree

(in/out)

3.72 3.70 5.39 6.17 288.41

Avg. path

length

4.06 4.43 3.63 3.98 1.72

Flows 6162 7527 654546 96057 249500

Min. link

bandwidth

2400000 1000000

Max. link

bandwidth

10000000 10000000

Min. ﬂow de-

mand

1 11 11 11 10

Max. ﬂow

demand

499374 396931 118476 147098 250000

TABLE III: Simulation parameters

Parameter Default value Range of values

|C1|0.450 0.250,0.350,0.450

|C2|0.350

θ0.850 0.750,0.850,0.950

α0.200 0.200,1.000

γ0.995

T106102,104,106

Fig. 2: Experimental network with eleven switches and ﬁve

hosts. The labels along the links indicate the port numbers of

the switches that they are connected to.

ﬁve hosts and eleven switches. These switches communicated

with a Ryu4OpenFlow controller. Each link had a capacity

of 100 Kbps. Eight ﬂows among the hosts were considered,

as shown in Table IV. Given this network and ﬂows, we

used SAFAR to allocate the bandwidth paths. Subsequently,

bandwidth slices were assigned to the unallocated ﬂows.

Thereafter, SAFAR generated the necessary forwarding rules

for the OVS switches. The corresponding queue creation

commands were also generated. Finally, we executed all these

commands to create the queues and install the forwarding

rules. We used Distributed Internet Trafﬁc Generator (D-ITG)

[41] to generate TCP ﬂows. Packets were generated for 60

seconds at a constant rate to match the ﬂow demands described

in Table IV. Each packet had a size of 100 bytes.

B. Performance Metrics

We considered the following metrics for evaluating the

performance of SAFAR:

•Fraction of ﬂows allocated: It is the ratio of the number

of ﬂows for which feasible bandwidth paths were deter-

4https://osrg.github.io/ryu/

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

9

TABLE IV: Flow paths under different allocation schemes (θ= 0.85).

Flow Demand Class 2-OPT 4-OPT SAFAR (cost = 0.2178)

(Kbps) (cost = 0.2173) (cost = 0.2020) Initial Final (with slicing)

h1→h3 50 C0s1→s3→s6→s9s1→s3→s6→s7→s9s1→s3→s6→s9s1→s3→s6→s9

h1→h4 90 C3s1→s4→s8→s10

h2→h4 25 C1s2→s5→s8→s10 s2→s5→s8→s10 s2→s4→s8→s10 s2→s5→s8→s10

h2→h5 20 C2s2→s4→s8→s11 s2→s4→s8→s11 s2→s4→s8→s11 s2→s4→s8→s11

h3→h1 50 C0s9→s7→s3→s1s9→s6→s3→s4→s1s9→s6→s3→s1s9→s6→s3→s1

h4→h1 12 C3s10 →s9→s6→s3→s1s10 →s8→s7→s3→s1s10 →s8→s4→s1s10 →s8→s4→s1

h4→h2 12 C3s10 →s8→s5→s2s10 →s8→s4→s5→s2s10 →s8→s4→s2s10 →s8→s5→s2

h5→h2 12 C2s11 →s8→s4→s2s11 →s8→s4→s2s11 →s8→s4→s2s11 →s8→s4→s2

80

82

84

86

88

90

92

94

96

98

100

N79 N87 N138 N315 N500

Percent of flows

Network

SPA-T Initial SAFAR

(a) Flows allocated

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

N79 N87 N138 N315 N500

Jain’s fairness index

Network

SPA-T

Initial

SAFAR

(b) Allocation fairness

0

100

200

300

400

500

600

700

800

N79 N87 N138 N315 N500

Processing time [s]

Network

SPA-T

Initial

SAFAR

0

10

20

30

40

50

N79 N87 N138 N315 N500

Processing time [s]

Network

(c) Processing times (Inset: pro-

cessing times of only SPA-T and

Initial)

0.132

0.134

0.136

0.138

0.14

0.142

0.144

0.146

0 500 1000 1500 2000 2500 3000

0.42

0.43

0.44

0.45

0.46

0.47

Cost

Jain’s fairness index

Time index

Cost

Fairness

(d) Cost evolution

Fig. 3: Performance of optimal ﬂow allocations.

mined to the total number of ﬂows, i.e., |FA|

|F|. Bandwidth

slice allocation is not considered here. We also consider

a variation of this metric by taking into account trafﬁc

classes, for example, percentage allocation of ﬂows with

delay-symmetric path requirements.

•Fairness of ﬂow allocation: This metric, measured by

(1), together with the previous one quantify the effec-

tiveness of a ﬂow allocation scheme.

•Processing time: This is the time taken by a ﬂow allo-

cation scheme to complete its operations. This includes

the time taken to compute the all-pair shortest paths, the

second shortest or alternative paths (only for SAFAR),

and the optimal bandwidth paths for the ﬂows.

•Throughput: It is measured as the number of bits re-

ceived per second by a host. It is used to quantify results

only from the experiment using OVS switches.

VII. RES ULTS

A detailed discussion on the experimental results are pre-

sented in this Section.

A. Flow Allocation Status

Fig. 3a shows the percentage of ﬂows allocated (y-axis)

by the three ﬂow allocation schemes for the ﬁve different

networks considered (x-axis). In general, SAFAR outper-

formed both SPA-T and Initial for all these networks. In

particular, a gap of about 9.6% can be observed between the

greedy allocations and SAFAR in case of N87 and N138.

This indicates that when alternative feasible bandwidth paths

exist, SAFAR is able to leverage such paths for better ﬂow

allocation. On the other hand, in case of N315, SPA-T and

Initial allocated about 98% of the ﬂows, whereas SAFAR

still offered an improvement by allocating more than 99% of

the ﬂows. A likely explanation is that, in N315, the primary

paths provided enough bandwidth to accommodate the ﬂows,

which potentially reduced the need for alternative paths even

if they existed. Finally, in N500, SAFAR was able to identify

feasible bandwidth paths for about 85% of the ﬂows, which

was relatively better than the percentage allocations by SPA-

T and Initial. It may be noted here that the link capacities

and ﬂow demands for N500 were generated randomly, which

likely impacted the availability of feasible bandwidth paths

and consequently, the number of ﬂows allocated. The results

also indicate that an increase in network size does not neces-

sarily mean that ﬂow allocation would also decrease; the link

capacities and ﬂow demands also affect the solution.

Fig. 3b plots the value of Jachieved under different

schemes. In general, SAFAR resulted in a higher value of Jas

compared to SPA-T and Initial. In other words, with SAFAR,

the average utilization (load) of links in a network improved.

In particular, in case N138, the fairness values obtained using

SPA-T and SAFAR, respectively, were 0.2585 and 0.3544. In

relative terms, this corresponds to about 37% improvement by

SAFAR. The considerable difference supports our earlier ob-

servation that SAFAR can suitably utilize alternative network

paths when available. Moreover, in case of N315 and N500,

SAFAR performed relatively better than SPA-T and Initial.

Fig. 3c shows the processing times of the different schemes.

It can be observed that SAFAR took considerably more

computation time than its counterparts, especially in case of

the very large network, N500, with huge number of ﬂow

demands5. However, the time spent by SAFAR was used to

gain performance improvements in terms of the number of

feasible bandwidth paths identiﬁcation and the fairness of link

utilization. Moreover, as the inset in Fig. 3c shows, even SPA-

T and Initial took signiﬁcantly higher computation time for

5The results indicate that SAFAR may not be an ideal candidate for real-

time or online execution in very large networks. In such scenarios, an initial

allocation may be quickly obtained using a greedy scheme, which can be

subsequently optimized using SAFAR. However, relocation of the C0ﬂows

may be skipped, so that the critical services are not disrupted.

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

10

0

0.2

0.4

0.6

0.8

N79 N87 N138 N315 N500

High priority delay-symmetric flows

Initial

SAFAR

0

3

6

9

12

N79 N87 N138 N315 N500

High priority flows

0

5

10

15

20

N79 N87 N138 N315 N500

Delay-symmetric flows

0

7

14

21

28

35

N79 N87 N138 N315 N500

Percent of flows

Network

Other flows

Fig. 4: Unallocated percentage of ﬂows under different trafﬁc

classes and network topologies.

N500 as compared to the smaller network topologies.

Fig. 3d shows how the cost of solution evolved for a partic-

ular experimental instance using the N79 network. It may be

observed that the cost did not decrease linearly, but ﬂuctuated

all the while, which demonstrates the working principle of SA.

A close look would reveal that such ﬂuctuations were more at

the beginning and much less toward the end. This is because,

the temperature Tdecreases with time, which reduces the

probability of accepting a more costly solution. The evolution

of ﬂow allocation fairness is also depicted in the Fig. (y-axis

on the right side). The Fig. shows that as SAFAR improved

the value of J, the overall cost as measured by (2) reduced.

Fig. 4 presents a closer look at the percentage of ﬂows under

different trafﬁc classes for which feasible bandwidth paths

could not be determined. We did not consider SPA-T here

because it does not associate the network ﬂows with any trafﬁc

class. The Fig. shows that both Initial and SAFAR were able to

allocate all ﬂows with high priority and delay-symmetric path

requirements (class C0) except for the N500 network topology.

Across the trafﬁc classes, the highest percentage of unallocated

ﬂows belong to the “others” class (C3), because both SAFAR

and Initial try to allocate as much ﬂows as possible from the

other three classes. On the other hand, across the network

topologies, the most unallocated ﬂows belong to N500. As

noted earlier, since the N500 network and ﬂow demands

were randomly generated, meeting all such ﬂow requirements

may not be practically feasible. Nevertheless, it still gives a

rough indication about how any ﬂow allocation scheme would

perform for very large networks.

Fig. 5 depicts how the unallocated ﬂow percentages varied

with the change in size of trafﬁc class. As the Fig. shows, when

the relative size of C1decreased from 0.45 to 0.25, more ﬂows

from this class were allocated due to the decreased competition

for bandwidth in class C1, as well as in C2. Moreover, with

|C2|ﬁxed, a decrease in |C1|implies an increase in |C3|. In

other words, more ﬂows from C3can be allocated because of

bandwidth availability, which is also captured by Fig. 5.

0

2

4

6

8

N79 N87 N138 N315 N500

High priority flows

|C1| = 0.25 |C1| = 0.35 |C1| = 0.45

0

6

12

18

N79 N87 N138 N315 N500

Delay-symmetric flows

0

10

20

30

N79 N87 N138 N315 N500

Percent of flows

Network

Other flows

Fig. 5: Variation in unallocated ﬂows with the relative size of

C1(|C2|= 0.35).

0

50000

100000

150000

200000

250000

300000

350000

400000

0 20 40 60 80 100 120 140

Bandwidth

Flow index

Demand

Allocation

Fig. 6: Bandwidth slices assigned to the unallocated ﬂows.

B. Bandwidth Slicing and Measurements

It may be recalled that slices of residual bandwidths are

allocated to ﬂows for which feasible bandwidth paths meeting

their demands cannot be found. Fig. 6 shows such bandwidth

slice allocation for a random scenario pertaining to the N87

topology. The Fig. shows that each ﬂow, otherwise unallocated

by SAFAR, got a positive share of bandwidth path between

their respective source and destination nodes. Of course,

such bandwidth slices did not always entirely match their

demand and there were wide divergences in some scenarios.

Nevertheless, low priority and non-critical services may still

beneﬁt from such an arrangement, where they get to send their

data at a reduced rate.

As noted in Sec. VI, we did an end-to-end implementation

by allocating ﬂows, assigning bandwidth slices, and creating

actual rules for switches (see Fig. 2). Table IV summarizes

the bandwidth paths allocated to the ﬂows by the different

schemes. The bandwidth demands of seven out of the eight

were met by 2-OPT, 4-OPT, and the initial solution of SAFAR.

The bandwidth slicing mechanism of SAFAR allocated a

path with 63.75 Kbps bandwidth to the h1→h4ﬂow.

SAFAR and 2-OPT had almost similar costs and allocated

similar paths except for two ﬂows. Although 4-OPT reduced

the cost further, the results indicate that, with higher values

of k, the average path length increases, which can affect

the end-to-end latency. Table V, on the other hand, shows

the OpenFlow ﬂow rules installed at the switches (only) for

the h1→h4ﬂow. The MAC addresses of h1and h4,

respectively, were 00:00:00:00:00:01 and 00:00:00:00:00:04.

When these switches received any packet matching these MAC

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

11

TABLE V: Rules installed for the h1→h4ﬂow.

Switch OpenFlow Forwarding Rules (some ﬁelds are omitted)

s1n_packets=4481, priority=101,in_port=5,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:04 actions=set_queue:1002,output:2

n_packets=18, priority=0 actions=CONTROLLER:65535

s4n_packets=4481, priority=101,in_port=1,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:04 actions=set_queue:1005,output:5

n_packets=0, priority=0 actions=CONTROLLER:65535

s8n_packets=4481, priority=101,in_port=1,dl_src=00:00:00:00:00:01,dl_dst=00:00:00:00:00:04 actions=set_queue:1005,output:4

n_packets=0, priority=0 actions=CONTROLLER:65535

s10 n_packets=7439, priority=101,dl_dst=00:00:00:00:00:04 actions=output:7

n_packets=9, priority=0 actions=CONTROLLER:65535

10

20

30

40

50

60

1-3 1-4 2-4 2-5 3-1 4-1 4-2 5-2

Throughput [Kbps]

Flow

Actual

Expected

Fig. 7: Throughput achieved by the ﬂows.

90

91

92

93

94

95

96

97

98

99

N79 N87 N138

Percent of flows

Network

T = 102

T = 104

T = 106

(a) Flows allocated

0

20

40

60

80

100

N79 N87 N138

Processing time [s]

Network

T = 102

T = 104

T = 106

(b) Processing time

Fig. 8: Effects of temperature on ﬂow allocation.

addresses, the packet was assigned to an appropriate queue of

the outgoing port. It may be recalled that these queues were

previously created, as indicated by Listing 1.

Fig. 7 shows the average throughputs achieved by the ﬂows

shown in Table IV. The experimentally obtained throughputs

were very close to the expected values computed by SAFAR.

The minor deviations are likely due to the emulation hardware

constraints and variations in packet generation rates, since no

packet drop was observed. This result, therefore, quantitatively

veriﬁes the bandwidth slicing feature of SAFAR.

C. Effects of Control Parameters

Fig. 8 illustrates how temperature Taffected the optimal

allocation of ﬂows. The Fig. shows that for any network, as T

increased, more ﬂows were allocated, in general. For example,

in N138, the ﬂow allocation improved from 93.05% to 94.98%

when Tincreased from 102to 104. However, at the same time,

the computational time taken also increased (Fig. 8b). This

is because, higher values of Tensured that the outer loop

in Algorithm 3 executed for a longer time. Our experiments

also indicated that an increase in the value of Timproved the

fairness of ﬂow allocation as well.

Fig. 9a shows that as the threshold of link utilization

increased, more ﬂows were allocated. This is because, with

more link bandwidth available, more ﬂows can be accommo-

dated. On the other hand, Fig. 9b shows that the fairness of

ﬂow allocation marginally decreased, in general. We did not

observe any noticeable impact of θupon the processing time.

90

91

92

93

94

95

96

97

98

99

100

N79 N87 N138

Percent of flows

Network

θ = 0.75

θ = 0.85

θ = 0.95

(a) Flows allocated

0

0.1

0.2

0.3

0.4

0.5

N79 N87 N138

Jain’s fairness index

Network

θ = 0.75

θ = 0.85

θ = 0.95

(b) Fairness of allocation

Fig. 9: Effects of link utilization threshold.

We also investigated the effects of the cost function for

different values of α. The Jain’s fairness index was found to

be relatively higher when αwas 1.0, as compared to when

it was 0.2. This follows from (2), where a higher value of α

puts more weight on the allocation fairness aspect. Moreover,

the processing times increased when αincreased. This is likely

due to the reason that, as αincreased, SAFAR tried to consider

more combinations of ﬂows in order to improve the fairness

and thereby, reduced the optimization cost.

VIII. CONCLUSION

Optimal trafﬁc ﬂow allocation is a hard problem, but a key

necessity for Industry 4.0, among others, to succeed, which

calls for a careful investigation of the problem by speciﬁcally

considering the typical characteristics of industrial networks.

To this end, our proposed scheme, SAFAR, near-optimally

allocates up to 99% of the ﬂows with different priorities

and delay-symmetry requirements, assigns bandwidth slices to

the remaining ﬂows, and ﬁnally, automatically generates the

necessary commands to be executed at the switches. In other

words, SAFAR provides an end-to-end solution for conﬁguring

new trafﬁc ﬂows and services for industrial networks. This, in

turn, opens up the possibility of integrating existing networks

with the emerging industrial IBNs ecosystem [2].

In the future, it would be interesting to reduce the average

processing time of SAFAR. Moreover, different deﬁnitions

of optimality and fairness, as well as additional business

requirements, can also be taken into account. For example,

teleprotection and other critical services require that there is no

single point of failure. Consequently, for each primary optimal

path, a backup or redundant path should be available. Finally,

the use of other metaheuristics may also be explored.

REFERENCES

[1] F. De Turck, P. Chemouil, R. Boutaba, M. Yu, C. E. Rothenberg, and

K. Shiomoto, “Guest Editors’ Introduction: Special Issue on Manage-

ment of Softwarized Networks,” IEEE Transactions on Network and

Service Management, vol. 13, no. 3, pp. 362–365, 2016.

Accepted Version (For Personal Use Only)

work in other works. [DOI: 10.1109/TNSM.2020.3035792]

12

[2] B. K. Saha, D. Tandur, L. Haab, and L. Podleski, “Intent-based Net-

works: An Industrial Perspective,” in Proceedings of the 1st International

Workshop on Future Industrial Communication Networks (FICN ’18).

New York, NY, USA: ACM, 2018, pp. 35–40.

[3] A. Amokrane, R. Langar, R. Boutaba, and G. Pujolle, “Flow-Based Man-

agement For Energy Efﬁcient Campus Networks,” IEEE Transactions on

Network and Service Management, vol. 12, no. 4, pp. 565–579, 2015.

[4] Y. Takahashi, K. Ishibashi, M. Tsujino, N. Kamiyama, K. Shiomoto,

T. Otoshi, Y. Ohsita, and M. Murata, “Separating Predictable and

Unpredictable Flows via Dynamic Flow Mining for Effective Trafﬁc

Engineering,” IEICE Transactions on Communications, vol. E101-B,

no. 2, pp. 538–547, 2018.

[5] N. Thazin, K. M. Nwe, and Y. Ishibashi, “End-to-End Dynamic Band-

width Resource Allocation Based on QoS Demand in SDN,” in 2019

25th Asia-Paciﬁc Conference on Communications (APCC), 2019, pp.

244–249.

[6] H. Karimianfard and H. Haghighat, “Generic Resource Allocation in

Distribution Grid,” IEEE Transactions on Power Systems, vol. 34, no. 1,

pp. 810–813, 2019.

[7] M. Koutensk´

y, V. Vesel´

y, and V. Maffıone, “Bandwidth-driven Flow

Allocation Policy for RINA,” in 2020 23rd Conference on Innovation in

Clouds, Internet and Networks and Workshops (ICIN), 2020, pp. 51–56.

[8] J. Wu, B. Cheng, M. Wang, and J. Chen, “Quality-Aware Energy

Optimization in Wireless Video Communication With Multipath TCP,”

IEEE/ACM Transactions on Networking, vol. 25, no. 5, pp. 2701–2718,

2017.

[9] S. Bera, S. Misra, S. K. Roy, and M. S. Obaidat, “Soft-WSN: Software-

Deﬁned WSN Management System for IoT Applications,” IEEE Systems

Journal, vol. 12, no. 3, pp. 2074–2081, 2018.

[10] S. Wang, Y. Ma, B. Cheng, F. Yang, and R. N. Chang, “Multi-

Dimensional QoS Prediction for Service Recommendations,” IEEE

Transactions on Services Computing, vol. 12, no. 1, pp. 47–57, 2019.

[11] J. Wu, C. Yuen, B. Cheng, Y. Shang, and J. Chen, “Goodput-Aware

Load Distribution for Real-Time Trafﬁc over Multipath Networks,” IEEE

Transactions on Parallel and Distributed Systems, vol. 26, no. 8, pp.

2286–2299, 2015.

[12] J. Wu, B. Cheng, M. Wang, and J. Chen, “Energy-Efﬁcient Bandwidth

Aggregation for Delay-Constrained Video Over Heterogeneous Wireless

Networks,” IEEE Journal on Selected Areas in Communications, vol. 35,

no. 1, pp. 30–49, 2017.

[13] R. Trestian, K. Katrinis, and G. Muntean, “OFLoad: An OpenFlow-

Based Dynamic Load Balancing Strategy for Datacenter Networks,”

IEEE Transactions on Network and Service Management, vol. 14, no. 4,

pp. 792–803, 2017.

[14] Y. Cheng and X. Jia, “NAMP: Network-Aware Multipathing in Software-

Deﬁned Data Center Networks,” IEEE/ACM Transactions on Network-

ing, vol. 28, no. 2, pp. 846–859, 2020.

[15] A. Karaagac, E. De Poorter, and J. Hoebeke, “In-Band Network Teleme-

try in Industrial Wireless Sensor Networks,” IEEE Transactions on

Network and Service Management, vol. 17, no. 1, pp. 517–531, 2020.

[16] B. K. Saha, S. Misra, and S. Pal, “SeeR: Simulated Annealing-Based

Routing in Opportunistic Mobile Networks,” IEEE Transactions on

Mobile Computing, vol. 16, no. 10, pp. 2876–2888, Oct 2017.

[17] B. P. R. Killi and S. V. Rao, “Capacitated Next Controller Placement

in Software Deﬁned Networks,” IEEE Transactions on Network and

Service Management, vol. 14, no. 3, pp. 514–527, 2017.

[18] Y. Hirano, F. He, T. Sato, and E. Oki, “Backup Network Design against

Multiple Link Failures to Avoid Link Capacity Overestimation,” IEEE

Transactions on Network and Service Management, pp. 1–1, 2019.

[19] S. Paul and Z. H. Rather, “A New Bi-Level Planning Approach to

Find Economic and Reliable Layout for Large-Scale Wind Farm,” IEEE

Systems Journal, vol. 13, no. 3, pp. 3080–3090, 2019.

[20] A. Sayin, E. G. Hoare, and M. Antoniou, “Design and Veriﬁcation

of Reduced Redundancy Ultrasonic MIMO Arrays Using Simulated

Annealing & Genetic Algorithms,” IEEE Sensors Journal, vol. 20, no. 9,

pp. 4968–4975, 2020.

[21] Z. Li and S. Rao, “The Inversion of One-Dimensional Soil Parameters

in the Frequency Domain With Considering Multilayered Earth Based

on Simulated Annealing Algorithm,” IEEE Transactions on Electromag-

netic Compatibility, vol. 62, no. 2, pp. 425–432, 2020.

[22] S. Layeghy, F. Pakzad, and M. Portmann, “SCOR: software-deﬁned con-

strained optimal routing platform for SDN,” CoRR, vol. abs/1607.03243,

2016.

[23] W. Sehery and C. Clancy, “Flow Optimization in Data Centers With

Clos Networks in Support of Cloud Applications,” IEEE Transactions

on Network and Service Management, vol. 14, no. 4, pp. 847–859, 2017.

[24] A. Sinha and E. Modiano, “Optimal Control for Generalized Network-

Flow Problems,” IEEE/ACM Transactions on Networking, vol. 26, no. 1,

pp. 506–519, 2018.

[25] B. K. Saha, L. Haab, and L. Podleski, “Flow Allocation in Industrial

Intent-based Networks,” in 2019 IEEE International Conference on

Advanced Networks and Telecommunications Systems (ANTS). IEEE,

2019, pp. 1–6.

[26] H.-D. Kim, J.-T. Kim, W.-H. Nam, S.-J. Kim, J.-Y. Choi, and B.-S.

Koh, “Irrigation Canal Network Flow Analysis by a Hydraulic Model,”

Irrigation and Drainage, vol. 65, no. S1, pp. 57–65, 2016.

[27] M. SteadieSeiﬁ, N. Dellaert, W. Nuijten, and T. Van Woensel, “A

metaheuristic for the multimodal network ﬂow problem with product

quality preservation and empty repositioning,” Transportation Research

Part B: Methodological, vol. 106, pp. 321–344, 2017.

[28] N. Saha, S. Bera, and S. Misra, “Sway: Trafﬁc-Aware QoS Routing

in Software-Deﬁned IoT,” IEEE Transactions on Emerging Topics in

Computing, pp. 1–1, 2018.

[29] S. Bera, S. Misra, and A. Jamalipour, “FlowStat: Adaptive Flow-Rule

Placement for Per-Flow Statistics in SDN,” IEEE Journal on Selected

Areas in Communications, vol. 37, no. 3, pp. 530–539, 2019.

[30] R. Jain, D. Chiu, and W. Hawe, “A quantitative measure of fairness

and discrimination for resource allocation in shared computer system,”

Digital Equipment Corporation, Technical Report 301, Sep. 1984.

[31] B. Dab, I. Fajjari, and N. Aitsaadi, “Online-Batch Joint Routing and

Channel Allocation for Hybrid Data Center Networks,” IEEE Transac-

tions on Network and Service Management, vol. 14, no. 4, pp. 831–846,

2017.

[32] A. Azzouni, R. Boutaba, and G. Pujolle, “NeuRoute: Predictive dynamic

routing for software-deﬁned networks,” in 2017 13th International

Conference on Network and Service Management (CNSM). IEEE, 2017,

pp. 1–6.

[33] I.-L. Wang, “Multicommodity Network Flows: A Survey, Part I: Ap-

plications and Formulations,” International Journal of Operations Re-

search, vol. 15, no. 4, pp. 145–153, 2018.

[34] T. Nishi, S. Akiyama, T. Higashi, and K. Kumagai, “Cell-Based Local

Search Heuristics for Guide Path Design of Automated Guided Vehicle

Systems With Dynamic Multicommodity Flow,” IEEE Transactions on

Automation Science and Engineering, vol. 17, no. 2, pp. 966–980, 2020.

[35] I. Maity, A. Mondal, S. Misra, and C. Mandal, “Tensor-Based Rule-

Space Management System in SDN,” IEEE Systems Journal, vol. 13,

no. 4, pp. 3921–3928, 2019.

[36] Q. Maqbool, S. Ayub, J. Zulﬁqar, and A. Shaﬁ, “Virtual TCAM for

Data Center switches,” in 2015 IEEE Conference on Network Function

Virtualization and Software Deﬁned Network (NFV-SDN), 2015, pp. 61–

66.

[37] J. Y. Yen, “Finding the K Shortest Loopless Paths in a Network,”

Management Science, vol. 17, no. 11, pp. 712–716, 1971.

[38] S. Ledesma, G. Avi˜

na, and R. S´

anchez, “Practical Considerations for

Simulated Annealing Implementation,” in Simulated Annealing, C. M.

Tan, Ed. Rijeka: IntechOpen, Sep. 2008, ch. 20, pp. 401–420.

[39] “Miniproject - Slicing Bandwidth,” https://sites.google.com/site/

slicingbandwidth/, 2014, Accessed: April 30, 2020.

[40] R. Hartert, S. Vissicchio, P. Schaus, O. Bonaventure, C. Filsﬁls,

T. Telkamp, and P. Francois, “A Declarative and Expressive Approach to

Control Forwarding Paths in Carrier-Grade Networks,” in Proceedings

of the 2015 ACM Conference on Special Interest Group on Data

Communication. New York, NY, USA: ACM, 2015, pp. 15–28.

[41] A. Botta, A. Dainotti, and A. Pescap `

e, “A tool for the generation

of realistic network workload for emerging networking scenarios,”

Computer Networks, vol. 56, no. 15, pp. 3531–3547, 2012.