Page 1
1
Computing Along Routes via Gossiping
Mehmet E. Yildiz, Student Member, IEEE, and Anna Scaglione, Senior Member, IEEE
Abstract
In this paper, we study a class of distributed computing problems where a group of nodes (destinations) is
interested in a function of data which are stored by another group of nodes (sources). We assume that the function
of interest is separable, i.e., it can be decomposed as a sum of functions of local variables, a case that subsumes
several interesting types of queries. One approach to solve this problem is to route raw information from the sources
to the interested destinations by either unicasting or multicasting. The second approach is to compute the function
of interest along some routes while propagating the information from the sources to the destinations. Considering
the second scenario, the goal of this paper is to examine how information should be forwarded to the intended
recipients, computing along the routes by gossiping with selected neighbors. Unlike efficient unicasting/multicasting
problems, nodes are interested in a specific function of the data, rather than the raw data themselves. Moreover,
unlike standard gossiping problems, in our case, the information needs to flow in a specific direction.
Given the underlying network connectivity and the source-destination sets, we provide necessary and sufficient
conditions on the update weights (referred to as codes) so that the destination nodes converge to the desired function
of the source values. We show that the evolution of the source states does not affect the feasibility of the problem,
and we provide a detailed analysis on the spectral properties of the feasible codes. We also study the problem
feasibility under some specific topologies and provide guidelines to determine infeasibility. We also formulate
different strategies to design codes, and compare the performance of our solution with existing alternatives.
Index Terms
Distributed computing, gossiping protocols, data aggregation, duplicate-sensitive data aggregation.
I. INTRODUCTION
A fundamental problem in networks is the transportation and distribution of information from one part of the
network to another [4]–[7]. We focus on a special network computation problem, where a group of destination
nodes is interested in a function that can be decomposed as a sum of functions of local variables stored by another
set of source-nodes. We refer to this problem as the Computing Along Routes (CAR) problem. This particular
problem has a wide range of applications including aggregation queries, distributed detection, content distribution,
decentralized traffic monitoring, distributed control and coordination [5], [8]–[10].
Mehmet E. Yildiz is with the School of Electrical and Computer Engineering of Cornell University, Ithaca, NY. Anna Scaglione
is with the Electrical and Computer Engineering Department of University of California at Davis. Corresponding e-mail addresses are
{mey7@cornell.edu} and {ascaglione@ucdavis.edu}. This work has been funded by National Science Foundation under NSF-ITR-CCR-
042827. Parts of this work was published in Asilomar 2008 [1], ITW 2009 [2] and CAMSAP 2009 [3].
Page 2
2
We focus on the case where the set of source nodes and the set of destination nodes are disjoint, and hence our
case does not include average consensus gossiping [5] as a special case. This particular assumption has strong
practical appeal since, in many cases, the nodes which are responsible for collecting the data and the nodes that
are designated to process these data are disjoint and geographically separated. For instance, in the problem of
distributed detection, only the fusion centers are interested in the outcome of the sensor decisions. In the case of
content distribution, the entity who is interested in the data does not necessarily have access to any part of it before
the distribution occurs. In the case of the leader-follower coordination problem [11], several nodes are to follow a
group of leaders, and the source and destination sets are clearly disjoint.
One solution to the problem we posed is to unicast between each pair of source-destination nodes via the shortest
path joining source and destination [12]. After receiving all the information, each destination node can compute
the desired function independently. Except in very special cases, this strategy is inefficient since the exact same
information flows in the network several times, and it is unreliable since it is severely affected by link failures.
The second approach is multicasting the information, having a single source node transmitting at a time to all
destinations, thereby allowing the computation of the sought results independently [13]. Thanks to network coding
[4], [14], [15], multicasting can be done using the links efficiently. However, this decomposition of the problem is
agnostic to the fact that the nodes do not want the data themselves, but an aggregate result.
In fact, the problem considered in this paper is closely related to the data aggregation and routing problems studied
in the computer science literature, where spatially distributed data is to be collected by a fusion center, utilizing
data aggregation and in-network processing techniques [8], [16]–[19]. The similarity is obvious if one observes
that solutions to the so called duplicate-sensitive data aggregation problem (see e.g. [17]) can be generalized to
find codes that solve our problem. Duplicate-sensitive data aggregation refers to the case where destinations seek
a single copy of data from each source. In the case of a single source and multiple destinations, the duplicate-
sensitive aggregation problem has been studied extensively, proposing energy efficient schemes based on spanning
trees [20], [21], and algorithms that are robust to link/node failures [17]. In the case of multiple source and multiple
destinations, solutions have been proposed including hierarchical structures and minimizing routes costs [18], [19],
[22]. These methodologies are not viable for solving the CAR problem, because one would need to explicitly
consider duplicating the data of all sources at the destinations. Moreover, it falls short of explaining the relationship
between type of topologies and the feasible queries and it does not incorporate feedback.
Observing the fact that the problem has two parts, i.e., the computation of the function and the routing of the
information, we are proposing a joint strategy where the computation of the desired function is achieved along the
routes via gossiping. Our approach incorporates feedback since gossiping based protocols are iterative, thus nodes
continuously exchange their values and create a feedback effect. We focus on separable functions, i.e., functions
Page 3
3
whose synopsis can be written as?
we will consider the scenarios where the function of interest is simply the average, since any other separable
ifi(xi), where the index i is over the set of nodes. Without loss of generality,
function can be calculated by initializing the source nodes’ values with states fi(xi).
As said above (see also Section VI), there are algorithms that result in partially directed solutions for the CAR
problem. However, one of the main interests of this paper is exploring the value of feedback.
Feedback is the trademark of Average Consensus Gossiping (ACG) protocols, which are considered attractive in
wireless sensor applications because the communications among nodes are limited to their immediate neighbors,
they are easily mapped into stationary asynchronous policies, while the network topology can be dynamically
switching. In ACG protocols each node in the network is both a source and a destination [5], [23], [24], [25].
An ACG protocol can directly be applied to our problem: in fact, non-source nodes could set their initial values
to zero, and via ACG the network would converge to a fraction of the average of the source nodes. When the
destination nodes know the number of source nodes and also the network size, the true average can be calculated
by rescaling the ACG result. However, this approach has two main drawbacks: 1) Since the desired information
has to be distributed to the whole network, the convergence will be slower, 2) Every node will have access to the
average at the end of the algorithm, and this is a major weakness since it does not preserve secrecy.
In summary, we are proposing a gossiping based algorithm for jointly routing and calculating the desired function
at the destinations. The innovative aspect of this procedure is that it incorporates feedback, unlike similar problems
that consider a unidirectional flow of data. On the other hand, it does not necessarily distribute the desired value
to the whole network which makes our scheme more secure and flexible.
In this paper, we limit ourselves to non-negative update weights (codes) and disjoint source and destination sets.
Under these assumptions, we formulate our problem in Section II. In Section III, we introduce necessary and
sufficient conditions for the existence of solutions to the prosed problem. In Section IV-A, we investigate spectral
properties of feasible codes and show what classes of code structures leads to feasible solutions. In Section IV-B, we
introduce reductions of the network topology which can be employed to simplify the design problem without loss
of generality. By focusing on stochastic codes, we provide necessary conditions on the topology for the feasibility
and discuss some infeasible cases in Section V. We introduce a formulation for so called partially directed solution
in Section VI, and discuss the complexity and communication cost of the algorithm in Section VII. We compare
the performance of our solution with the existing solutions in Section VIII, and conclude our paper in Section IX.
We note that a setup resembling more to our problem is considered in [26], where the authors have applied the
gossiping algorithm to solve a sensor localization problem, assuming that each node in the network wants to
compute a linear combination of the anchor (source) nodes, to determine their exact locations. Unfortunately, this
analysis can not be utilized to solve our problem since all of the non-source nodes are destinations, and destination
Page 4
4
nodes are not necessarily interested in the same function of the source nodes. A significant work is also due to
Mosk-Aoyama et.al. who have considered distributed computation of separable functions as well as information
dissemination on arbitrary networks [27], [28]. Morever, Benezit et.al. have studied average consensus problem via
randomized path averaging to achieve increased convergence rates [25]. However, our problem differs from these
models in the sense that source and destination sets are disjoint, and there may exist intermediate nodes which are
neither sources nor destinations.
II. PROBLEM FORMULATION
In this paper, we consider a connected network (N,E) with N nodes and the corresponding edge set E which
consists of ordered node pairs (i,j). Given the edge (i,j), i is the tail and j is the head of the edge. We define
the neighbor set of node i as Ni? {j ∈ {1,2,...,N} : (j,i) ∈ E}.
We consider the following problem setup: Each node in the network has an initial scalar measurement denoted by
xi(0) ∈ R where i ∈ {1,...,N}. Let S ? {1,2,... ,N}. There is a set of nodes (destination nodes), denoted as
SD⊆ S, which are interested in the average of a set of nodes (source nodes), denoted as SS⊆ S. We note that
SS∩SD= ∅, i.e., a source node cannot be a destination node. We restrict our attention to a synchronous gossiping
protocol, with constant update weights:
xi(t + 1) = Wiixi(t) +
?
j∈Ni
Wijxj(t), i ∈ S,
(1)
where t is the discrete time index, Wijis the link weight corresponding to the edge (j,i). We note that if j ?∈ Ni,
then Wij= 0. At each discrete time instant, each node updates its value as a linear combination of its own value
and its neighbors values. This type of analysis is usually the first step to investigate more flexible asynchronous
solutions [29]. If we define an N × N matrix W such that [W]ij= Wij and x(t) = [x1(t),x2(t),...,xN(t)]T,
then (1) can be written in the matrix form as:
x(t + 1) = Wx(t) = Wt+1x(0).
(2)
Parallel to the network coding literature, we refer to the matrix W as the code that we are interested in designing.
The equation above implies that:
lim
t→∞x(t) = lim
t→∞Wtx(0) = W∞x(0)
where W∞? limt→∞Wt(assuming that limt→∞Wtexists). One way to solve our problem is assigning zeros
as initial state values for non-source nodes, and running an average consensus algorithm on the network. The
consensus value will be a rescaled version of the source nodes’ average; thus, to determine the desired function,
Page 5
5
the destination nodes have to multiply the consensus value by a scaling factor. For this approach, any code W
that solves the average consensus can be employed [24], [29]–[31]. This approach has two major disadvantages:
First, since the whole network has to converge to a consensus, the converge will be slow which will require more
resources, i.e., energy, time (c.f. Section VIII). Second, at the end of the algorithm, every node will have access to
the average, and this is a major weakness in the case of secret communications.
The approach which we pursue here is to design a W that produces the desired computation irrespective of what
x(0) is, and distributes the value to the destinations only. Since, we are only interested in calculating the average
of the nodes values in SSat all j ∈ SD, we have the following inherent constraints on W∞:
W∞
jk=
|SS|−1
k ∈ SS
0k ?∈ SS
, ∀j ∈ SD,
(3)
where |.| denotes the cardinality of its argument. In other words, the entries of the limiting weight matrix should
be chosen such that destination nodes multiply source nodes’ values by |SS|−1(for averaging), and multiply non-
source nodes’ values by 0. Therefore, the rows of W∞corresponding to destination nodes should have |SS|−1for
the source node columns, and 0 otherwise.
Let us denote the structure above, imposed by sets SDand SS, as F(SD,SS), and as F(E), the structure imposed
by the network connectivity, i.e., Wij= 0 if (i,j) ?∈ E. We denote a code W, which satisfies (3), as an Average
Value Transfer (AVT) solution to the problem of computing along routes (CAR). The following definitions are in
order:
Definition 1. A code W is a feasible AVT solution for CAR on a given network (N,E) and source-destination sets
(SD,SS), if and only if W ∈ F(E) and W∞∈ F(SD,SS) with W∞< ∞.
Definition 2. CAR is AVT infeasible, if there does not exist any AVT solution.
We note that < denotes elementwise strict inequality. In the rest of the paper, unless stated otherwise, we will
omit the word AVT and refer to Definition 1 as the solution and 2 as the feasibility condition. Therefore, our
(in)feasibility definition considers only AVT solutions.
In the rest of the paper, we limit ourselves to codes in the set of nonnegative matrices, i.e., W ≥ 0 and ≥ represents
elementwise inequality, in analogy with the ACG policies [29]. In the following section, we will introduce necessary
and sufficient conditions for a W matrix to be an AVT solution for the CAR problem.
III. NECESSARY AND SUFFICIENT CONDITIONS ON FEASIBLE CODES
In the following, we first give necessary conditions on feasible W and W∞in addition to the structure given in (3).
We will then use the resulting equations to introduce necessary and sufficient feasibility conditions.
We first partition N sensors in the network into three disjoint classes: M sensors that belong to the source nodes set,
Page 6
6
K sensors that belong to the destination nodes set, and L sensors that belong to neither source nodes nor destination
nodes (called intermediate nodes). In other words, |SS| = M, |SD| = K and L = N − M − K. Without loss of
generality, we index the set of source nodes as {1,...,M}, the set of destination nodes as {M +1,...,M +K},
and the set intermediate nodes as {M + K + 1,...,N}. At this point, we can partition the N × N matrix W as:
W =
ADG
BEH
CFP
(4)
such that A ∈ RM×M, E ∈ RK×Kand P ∈ RL×L. Assuming that limt→∞Wtexists, we denote this limit as W∞
and its partitions using the superscript?, i.e., A?,D?,G?, etc. Since the set SDis only interested in the average of
the set SS, then B?= 1/|SS|11Tand E?= H?= 0 by (3). We note that 1 is the all ones vector of the appropriate
dimensions. This inherent structure results in the following necessary constraints on W and W?:
Lemma 1. Given a network F(E) and the sets SSand SD, if W ≥ 0 is a feasible solution to CAR, then:
1) D?= G?= 0,
2) D = G = 0,
3) 1TA = 1T.
Proof: The proof of the lemma is given in Appendix A.
Lemma 1 shows that the information flow between the sources and the rest of the network must be one-way, i.e.,
from the sources to the network. Such a finding is not surprising since the network is only interested in the average
of the source nodes, and the average will be biased if the sources mix their state values with values from the
non-source nodes, whose initial states are arbitrary. Lemma 1 also shows that the row sums of the matrix A are all
equal to 1. Since the matrix A governs the communication among the source nodes, the sum of the source values
must remain constant through the iterations. This result is also intuitive since, otherwise, the average of the source
nodes would change and W would not be a solution for the problem. The fact that the flow from the source nodes
to the rest of the network is directional, has important consequences on what designs are feasible, as we will see
next.
In light of Lemma 1, we will focus on the network codes which have the structure:
W =
A00
BEH
CGP
.
At this point, for mathematical brevity, we repartition the W matrix in four super-blocks recycling the previous
Page 7
7
symbols as follows:
W =
A0
BD
where B =
B
C
,D =
EH
GP
.
(5)
Of note is that, the new B ∈ RN−M×Mand D ∈ RN−M×N−M. Similarly, we can partition the state vector x(t):
s(t) = [x1(t),...,xM(t)]T,r(t) = [xM+1(t),...,xN(t)]T.
We note that s(t) represents the evolution of the source states while r(t) represents the behavior of the rest of
the network including the destination nodes. Similarly, the matrix A governs the communication among the source
nodes, and the matrix D determines the communication structure among the rest of nodes. B instead governs the
flow from the sources to the rest of the network. We expand the general form of the update in (1) using (5) as:
s(t + 1)=As(t),
r(t + 1)= Bs(t) + Dr(t).
The equations above can be rewritten in terms of the initial conditions as:
s(t + 1) = At+1s(0),
r(t + 1) = Dt+1r(0) +
t
?
l=0
Dt−lBAls(0).
The linear system of equations in a compact form is:
x(t + 1) =
s(t + 1)
r(t + 1)
= Wt+1x(0) =
At+1
0
?t
l=0Dt−lBAl
Dt+1
s(0)
r(0)
.
(6)
At this point, we can state the main theorem of the paper which introduces the necessary and sufficient conditions
on the feasible network codes W ≥ 0:
Theorem 1. Given a network F(E) and the sets SSand SD, a W ≥ 0 matrix of the form (5) is an AVT solution
to the CAR problem if and only if:
1) W is in the form of:
W =
A0
BD
,
(7)
where A ∈ RM×M, D ∈ RN−M×N−Mand M = |SS|,
2) limt→∞Wtexists and is finite,
3) limt→∞
?Dt+1?
1:K= 0,
Page 8
8
4) limt→∞
??t
l=0Dt−lBAl?
1:K=
1
M11T,
where [.]1:Kdenotes the first K rows of its argument, and K = |SD|.
Proof: While sufficiency follows from (6) and the problem definition given in Section II, these conditions are
necessary because of Lemma 1 and equation (6).
In the following, we show that one can simplify the fourth constraint in Theorem 1 by exchanging the powers of
A with its limit, i.e., the evolution of the source states does not affect the feasibility of the solution.
Lemma 2. Assuming that conditions (1) − (3) in Theorem 1 hold, then condition (4) holds if and only if
lim
t→∞
?
t
?
l=0
Dt−lBA∞
?
1:K
=
1
M11T,
(8)
where A∞= limt→∞At.
Proof: The proof is given in Appendix B.
Lemma 2 shows that if there exists a limiting matrix A∞which satisfies the constraint in (8), then any A matrix such
that limt→∞At= A∞, satisfies the fourth constraint in Theorem 1. Therefore, given the underlying connectivity,
one can assume that A has already converged to its limit A∞, and then seek a solution for partitions B and D.
Unfortunately, even under this simplification, designing a feasible W is difficult, since the fourth constraint in
Theorem 1 is non-convex with respect to the elements of the partitions D, B and A∞, and also consists of all
non-negative powers of D. Moreover, it is not clear from Theorem 1 what kind of topologies do or do not have
solutions for the CAR problem we posed.
In the following sections, we will both try to simplify the design of an AVT code and determine topologies where
the CAR problem is known to be (in)feasible.
IV. SPECTRAL ANALYSIS AND TOPOLOGY REDUCTIONS
A. Spectral Analysis of Feasible Codes
We start our discussion by introducing some necessary definitions. Motivated by the theory of Markov Chains, we
say that node i has access to node j, if for some integer t, Wt
ij> 0. Two nodes i and j, which have access to
the other, are said to be communicating. Since communication is an equivalence relation, the set of nodes which
communicate forms a class. A class which consists of only source nodes is called a source class. In the following,
we also introduce the graph theoretic definition of a source class.
Definition 3. A set of source nodes forms a source class if and only if the sub-graph consisting of these source
nodes only is irreducible, and including any other source nodes results in a reducible sub-graph.
We note that two definitions are equivalent. At this point, we are ready to introduce our first condition on the rank
of W∞.
Page 9
9
Lemma 3. Given a feasible W which satisfies Theorem 1, the following holds true:
# of source classes ≤ rank(W∞) ≤ # of classes.
Proof: The proof is given in Appendix C.
Let’s denote rank(W∞) as rW∞ and U as the set of linearly independent columns of W∞, implying that |U| = rW∞.
If we denote elements of U as ui, 1 ≤ i ≤ rW∞, then W,W∞satisfy:
Wui= W∞ui= ui, ∀ i ∈ {1,...,rW∞}.
(9)
In other words, the linearly independent columns of the limiting matrix are the dominant eigenvectors of W matrix,
i.e., the eigenvectors corresponding to the eigenvalue one. Therefore, understanding the column structure of the
limiting matrix is crucial to determine the spectral properties of a feasible W and vice versa.
We first note that each source class SC will be represented by a single dominant eigenvector uiin U, which is
equal to [W∞]i ∀i ∈ SC, where [.]idenotes the i-th column of its argument. We note that W∞
kiis the weight of
node i at node k, in the limit. For a given source node i, if W∞
ki> 0, then node k has access to the weighted value
of node i. It is clear that if k is a destination node, then W∞
ki= 1/M, where M is the number of source nodes.
The argument follows from Theorem 1, since W is feasible.
Let’s assume that rW∞ is strictly greater than the number of source classes. Therefore, there exists at least one
class C which is not a source class and ρ(WC) = 1, where WCis the sub-matrix of rows and columns of W that
correspond to the set C. If this is the case, the columns of W∞corresponding to the elements of the class must
have some non-zero values. Thus, there exists at least one i ∈ C, where W∞
ki> 0 ∀k ∈ C. In other words, some
of the nodes in the network will have access to the weighted values of the nodes in class C in the limit. However,
these nodes cannot be utilized in calculating the average of the source nodes at the destinations. In other words,
if there exists a node k which has access to a non-source class C in the limit, the destination nodes cannot have
access to the node k in limit, since otherwise the third condition in Theorem 1 would not hold. We will summarize
result of the discussion above by the following remark:
Remark 1. For a feasible code W and a non-source class C, if ρ(WC) = 1, then removing class C from the
network does not change the feasibility of the AVT algorithm.
Another way to interpret this condition is the following: If an AVT solution exists for a given scenario, then there
exists at least one feasible solution under the extra constraint that there is no non-source class whose corresponding
sub-matrix has spectral radius one. In other words, one can pose the extra constraint that all of the non-source
classes have sub-matrices with spectral radii strictly less than 1, and such constraint does not change the feasibility
of the problem. We also note that since sub-matrices corresponding to the non-source classes are also sub-matrices
Page 10
10
of the partition D of W (since D governs communications among non-source nodes), the eigenvalues of non-
source classes are also the eigenvalues of D. Therefore, the extra constraint that we posed on non-source classes
is equivalent to ρ(D) < 1. At this point, we would like to discuss the physical significance of ρ(D) < 1. We
first note that ρ(D) < 1 implies limt→∞Dt= 0. Moreover, as we have mentioned above, the partition D governs
the communication among non-source nodes. Therefore, in this particular case, non-source nodes will have zero
information about themselves in the limit. Such a result is not surprising since none of these nodes are interested
in the values of non-source nodes. On the contrary, a subset of these nodes (destination nodes) are interested in the
average of the source nodes, while the intermediate nodes do not aim at receiving anything at all. We summarize
our result in the following lemma:
Lemma 4. There exists a feasible code W which satisfies Theorem 1 if and only if there exists a W?which satisfies
both Theorem 1 and ρ(D) < 1.
While adding one more constraint to Theorem 1 may seem like increasing the complexity of an already difficult
problem, we observe the following: A feasible W has to satisfy rW∞ linear equations of the form (9). Moreover,
ρ(D) < 1 implies that the rank of W∞will be equal to its lower bound given in Lemma 3. This, in return,
implicitly shows that a feasible code W has to satisfy fewer equality constraints.
Moreover, since ρ(D) < 1, limt→∞Dt= 0, thus the third condition in Theorem 1 is automatically satisfied. The
condition in Lemma 2 becomes:
?
lim
t→∞
t
?
l=0
Dt−lBA∞
?
1:K
=?[I − D]−1BA∞?
1:K=
1
M11T,
where I is the identity matrix with appropriate dimensions. Therefore, the fourth condition in Theorem 1 is also
simplified.
At this point, we will introduce our last condition on the number of source classes in the network:
Lemma 5. Consider a CAR problem with the sets SS,SD, a network F(E), and a code W with ρ(D) < 1. If W
is a feasible solution, then the following must hold:
# of source classes ≤ 1 + N − K − rank(B).
Proof: The proof is given in Appendix D.
We note that K is the number of destination nodes and the matrix partition B governs the communication between
source nodes and the rest of the network. Lemma 5 shows that if a network code W is feasible, then the number
of source classes and the rank of the partition B have to balance out each other, i.e., their sum has to be less
than or equal to 1 + N − K. Therefore, network topologies which do not satisfy Lemma 5 will be infeasible. In
the following, we will illustrate our point with an example. Fig. 1 shows a network with two sources and two
Page 11
11
S1S2
D1 D2
Fig. 1.
destinations. Each source node forms a source class by itself, therefore there are two source classes in the network.
An infeasible scenario with two sources and two destinations.
By Lemma 5, the partition B of W has to satisfy: rank(B) ≤ 1. But, the rank of B can be set to one if and only
if one of the two links going from the sources to the destinations is removed. This is not possible since one of the
sources will be unconnected to the destinations. Therefore, there does not exists any feasible AVT code for this
network.
In the following sections, we will propose several claims under the assumption that ρ(D) < 1. In light of Lemma 4,
such statements do not change the generality of the claims, since our assumptions do not change feasibility of AVT
codes for a given problem.
B. Reduction of Topology to Study Feasibility
In this section, we will introduce methods to perform topology reduction, while leaving the feasibility of AVT
codes unchanged, by utilizing our discussions in Sections III and IV-A. In return, we will simplify our design
problem and develop tools with which we can determine, by inspection, the topologies for which AVT codes are
(in)feasible.
In Definition 3, we have given the definition of source class for a given code W. However, for a given network
F(E) and source set SS, source classes are not unique, i.e., one may change the structure of the source classes by
changing the entries of W. An example is given in Fig. 2. Both networks have the same underlying connectivity
F(E). The first network is induced by a W which assigns non-zero weights to the links between source nodes.
Thus, two source nodes form a single source class. On the other hand, the second network corresponds to a W
which assigns zero weights to the links between source nodes. Since these nodes cannot communicate, each forms
a source class by itself.
In Section IV-A, we have also argued that the rank of W∞is closely related with the complexity of the problem
and, under ρ(D) < 1, the rank is equal to the number of source classes in the network. Then, one can also argue
that for CAR, one should cluster source nodes as much as possible, thus reduce the complexity of the problem.
The definition of the minimal source class follows:
Definition 4. For a given network and source set, the minimal source class is such that the total number of source
Page 12
12
S1S2
S1 S2
Network 1Network 2
Fig. 2.
classes is minimized.
Two networks corresponding to the same underlying connectivity but two different network codes, W1 and W2.
This is the case where all of the existing links between source nodes are assigned non-zero weights. The number of
source classes in the minimal class can be obtained by removing all but the source nodes from the network F(E),
and counting the irreducible sub-graphs that remain.
Next, we will prove that focusing on the minimal class does not change the feasibility of the problem, hence we
will conclude that, combined with the design advantages given above, one can focus on minimal source class codes.
Lemma 6. Consider a problem F(SS,SD) and a network F(E). If the problem is feasible, then there exists at
least one solution W which utilizes the minimal source class.
Proof: The proof is given in Appendix E.
The key point of the proof is that activating links among sources without changing an existing solution does not
affect the feasibility of AVT codes. From Lemma 6, the first topology reduction is in order:
Reduction 1. Use the minimal source class and utilize average consensus algorithm within each class.
We note that utilizing average consensus algorithm within each class does not affect the feasibility of our problem.
Such an observation follows from the fact that for a given feasible W, source node i must have equal information
about the source nodes in its class, i.e., W∞ij= αi, ∀j ∈ SC?, where SC? is the class to which node i belongs.
This observation is due to Lemma 1 and since each source class converges to a rank one matrix (c.f. Appendix C).
Therefore one can construct a feasible W?where each class utilizes an average consensus algorithm, by employing
an ACG algorithm among the sources and rescaling some of the entries of the partition B of W matrix, keeping
the partition D unchanged.
As a result of this reduction, each node in a given source class converges to the average of the initial source nodes
in that particular class.
Reduction 2. Treat each source class as a single node whose value is equal to the average of the source nodes
in that particular class. Connect all non-source nodes, which are adjacent to the source nodes in the class, to this
single node.
Page 13
13
We note that the second reduction does not change the feasibility of the AVT code due to Lemma 2.
Reductions 1-2 combined with ρ(D) < 1 simplify our design problem significantly, with respect to the conditions
given in Theorem 1, without changing the feasibility of the problem.
V. A NECESSARY CONDITION ON THE TOPOLOGY
In this section, we focus on AVT codes which satisfies W1 = 1 as well as Theorem 1. Since such condition
combined with non-negativity, implies that W is a stochastic matrix, we will refer these codes as stochastic codes.
In return, we will provide a non-algebraic necessary condition for the existence of a feasible solution, which is easy
to interpret. At the end of the section, we will discuss topologies which are known to have no feasible solutions.
Before introducing the main result of the section, we would like to emphasize the close relationship between
stochastic AVT codes and non-homogenous random walks on graphs with absorbing states. A stochastic AVT
code W represents the transition matrix of a Markov Chain M whose state space consists of the nodes in the
network (N,E). The structure of the chain (locations of possible non-zero transition probabilities) is defined by
the underlying connectivity F(E), i.e., Wijis the probability of jumping from state i to state j in a single step.
Moreover, due to the structure of W in (7), each source class forms an absorbing class by itself. For a given
non-source node i and a source class SC?, the quantity?
is absorbed by the source class SC? given the fact that M has been initialized at node i [32]. Moreover, for a given
j∈SC?W∞
ijwill be equal to the probability that the chain
source node i ∈ SC? and j ∈ SD, the quantity W∞
ij/?
j∈SC?W∞
ijis the frequency that the chain visits node i
given the fact that M has been initialized at node j and it has been absorbed by the source class SC? [32]. Given
the discussion above, we conclude the following:
Remark 2. Constructing a stochastic AVT code is equivalent to designing a transition probability matrix W for a
Markov chain {M(t)}∞
t=0on graph (N,E) with state space S, where each source class forms an absorbing class.
Moreover, for each destination node j ∈ SDand each source class SC?, absorbtion probabilities should be chosen
such that:
P(M(∞) ∈ SC?|M(0) = j) =|SC?|
M
,
where P(.) the probability of its argument. Moreover, for each source node k ∈ SC?,
P(M(∞) ∈ k|(M(∞) ∈ SC?|M(0) = j)) =
1
|SC?|.
Unfortunately, the formulation given above does not simplify our design problem, since constructing transition
probability matrices for complex chains with given stationary distributions is an open problem in the literature.
However, we believe that pointing out the equivalence relation is necessary as it brings a different perspective to
Page 14
14
S1D1S2D2
Fig. 3.
the AVT problem. Moreover, we will make use of the equivalence in the proof of the following lemma:
Line network topology. S and D denotes source nodes and destinations nodes respectively.
Lemma 7. Consider a network F(E) and the sets SSand SD. Partition the network into two disjoint sets (P,Pc)
such that there exists at least one source class-destination pair in both sides of the network. For a given partition,
we denote the links going from one set to the other set as cut edges, i.e, an edge (i,j) is a cut edge if i ∈ P and
j ∈ Pc, or j ∈ P and i ∈ Pc. If there exists a feasible stochastic code, then there exits at least two (cut) edges
between P and Pcfor all such partitions.
Proof: Proof of the lemma is given in Appendix F.
Unlike the results we have proposed in the previous sections, Lemma 7 gives us a topology based method to detect
infeasible cases. We note that the condition given in the lemma has to be satisfied by a network where stochastic
AVT codes are feasible, besides the connectivity constraint that we imposed. Therefore, one can conclude that the
connectivity assumption is not sufficient for the existence of a solution under stochastic codes. At this point, we
remind our readers that the connectivity assumption is a sufficient condition for the existence of an ACG solution [5].
Thus, we conclude that demands on stochastic AVT codes are stricter than the ones for the consensus problems.
We would like to note that, as stated in the hypothesis, the conditions in Lemma 7 is valid for the cases where there
are at least two source classes in the network. For the scenarios where there is only one source class, Lemma 7 is
not valid.
In the following, we will give examples of infeasible network topologies for CAR under stochastic AVT codes. We
will be considering nontrivial CAR instances with more than one source and destination nodes.
1) Line network topology: A line network topology is given in Fig. 3. The vertical line shows a cut where
both partitions have one source-destination pair. By Lemma 7, such a network does not have a solution for
CAR, since the cut shown in the figure has a single cut edge.
2) Mesh network topology with a bottleneck link: The network is shown in Fig. 4. It does not have a solution,
since there is a single cut edge for the given partition.
VI. CONSTRUCTION OF PARTIALLY DIRECTED AVT SOLUTIONS
As we have discussed in the preceding sections, it is difficult to construct feasible AVT codes due to the non-linearity
of the constraints in Theorem 1. For this reason, we have proposed several simplifications, which do not affect
the feasibility of an AVT code in Sections IV-A and IV-B. While these simplifications are valuable in determining
whether AVT codes are feasible or infeasible, they did not lead to a constructive way for designing AVT solutions.
Page 15
15
S1
D1
S2
D2
Fig. 4.
In the following, we will formulate an integer programming problem whose solution will be utilized to construct so
Mesh network topology. S and D denotes source nodes and destinations nodes respectively.
called partially directed AVT solutions. These solutions belong to a subset of the AVT solutions given in Theorem 1.
We first introduce the definition of a partially directed AVT solution:
Definition 5. Consider a network F(E) and the sets SSand SD. A code W is a partially directed AVT solution,
if it satisfies Theorem 1 and each link on the network, except the links among the source nodes, can be utilized
only in one direction, i.e., WijWji= 0, ∀ i,j ?∈ SS.
It should be clear from Definition 5 why these codes are called partially directed solutions, i.e., communication is
directed only among the non-source nodes. Before proposing our construction, we will introduce some necessary
definitions.
Since the communications among source nodes are bidirectional, we can employ Reduction 2 in Section IV-B
and assume that each source class has already converged to the average value of its members. We enumerate the
source classes (arbitrarily ordered) and we define the set of source classes as SC. We also define E?⊂ E which
contains all edges except the edges among the source nodes. For any given U ⊂ S, we define a set of edges
E(U) = {(i,j) ∈ E?|i,j ∈ U}. In other words, E(U) is the set of edges whose end points belong to the set U.
In the following lemma, we introduce our method to construct partially directed AVT codes:
Lemma 8. Consider a network F(E) and the sets SSand SD. For k ∈ SCand l ∈ SD, we define a variable bkl
i
as:
bkl
i=
1,
if i = k,
−1,
if i = l,
0,
otherwise.
Consider the following integer programming formulation:
miminize
max
k∈SC,l∈SD
?
(i,j)∈E?
zkl
ij,
(10)
subject to
?
(i,j) ∈ E?, k ∈ SC, l ∈ SD,
j|(i,j)∈E?
zkl
ij−
?
j|(j,i)∈E?
zkl
ji= bkl
i
i ∈ {SC∪ {M + 1,...,N}},k ∈ SC,l ∈ SD,
(11)
zkl
ij≤ uij,
(12)
Page 16
16
uij+ uji≤ 1,(i,j) ∈ E?,
(13)
?
(i,j)∈E(U)
uij≤ |U| − 1,U ⊂ N,U ?= ∅,
(14)
zkl
ij∈ {0,1},(i,j) ∈ E?, k ∈ SC, l ∈ SD.
(15)
If the integer program given above has a solution, namely, z?kl
ij, (i,j) ∈ E?, k ∈ SC, l ∈ SD, we define yk
ijas
follows:
yk
ij=
1,
if
?
l∈SDz?kl
ij≥ 1,
0,
otherwise.
Then, a feasible partially directed AVT code can be constructed as follows:
Wji=
?
k∈SC|SC(k)|yk
?
ij
?
l∈Nj
k∈SC|SC(k)|yk
lj,
if
?
l∈Nj
?
k∈SC|SC(k)|yk
lj?= 0 and (i,j) ∈ E?,
1
|Nj|+1,
if i,j ∈ SSand i ∈ Nj, or j ∈ SSand i = j,
0,
otherwise,
(16)
where SCkis the set of nodes which belongs to the k-th source class.
Proof: The proof is given in Appendix G.
The integer programming formulation given in the lemma is a directed multicommodity flow problem with acyclicity
constraint [33]. In particular, one can map the variable bkl
ito the net inflow at node i of data with origin k and
destination l. The value of the net inflow is positive at the sources, negative at the destinations, and zero otherwise.
The variable zkl
ijindicates the amount of information with origin k and destination l that flows through link (i,j).
yk
ijis equal to one if there exists at least one flow on (i,j) that is originated from source class k.
We note that the constraint given in (11) guarantees the flow is preserved at each node, i.e., the total inflow to
a given node is equal to the total outflow from this node. Moreover, uij is a binary variable and equal to one if
there exists at least one flow which utilizes the link (i,j) (12). Otherwise, it is equal to zero. Hence, (13) is the
one-way flow constraint. Finally, (14) guarantees the flows are acyclic. The objective function of the problem is the
maximum of number of paths between all source-destination pairs, thus the problem is minimizing the convergence
time of the directed part of the algorithm.
An example is given in Fig. 5. There are 4 source nodes with 3 source classes and 2 destination nodes. In Fig. 5(a),
red, blue and green arrows represent flows from source classes to the destination nodes. The corresponding link
weights are shown in Fig. 5(b). We note that the weight of the link connecting the flows from nodes 1 and 2 to the
central hub is twice as much as the link weights of the other flows. This is because the number of source nodes in
that particular flow is twice as much as the size of nodes in other flows.
Page 17
17
D1D2
S2
S1
S3
S4
(a) Optimal flows.
S2
S3
D1 D2
S4
S1
(b) Corresponding link weights.
Fig. 5.
We would like to conclude the section with the following observation: Existence of a partially directed AVT solution
A directed solution. S and D represents source and destination nodes respectively.
is a sufficient condition for the existence of an AVT solution, since partially directed AVT solutions also satisfy
Theorem 1. On the other hand, the reverse argument may not always be true, i.e., existence of an AVT solution
does not imply the existence of a partially directed AVT solution. Remarkably, we were not able to find a counter
example; we conjecture that the condition is both necessary and sufficient.
We also note that directed solutions are less robust to link/node failures while undirected solutions are more robust
due to the presence of feedback. For instance, if a link on the path between a source-destination pair fails in a
partially directed solution, the destination node will receive no information about that particular source . In the
case of undirected solution, instead destination nodes may not converge to the desired average, but still have partial
information about the source.
VII. COMPLEXITY AND COMMUNICATION COST OF AVT CODES
In this section, we will first briefly discuss the complexity of constructing AVT codes. We, then, analyze the
communication cost (number of message exchanges) of AVT codes on random geometric graphs.
A. Complexity
The problem formulation given in Theorem 1 is non-convex, thus closed form solutions do not exists in general.
On the other hand, existing numerical methods for systems of nonlinear equations can be utilized to determine
feasible solutions for a given problem [34]. We note that, while these methods perform fairly well in practice,
convergence to a true solution is not guaranteed (because of possible singular Jacobians through the iterations, a
wrong initialization point, etc). On the other hand, the partially directed AVT solution is an integer programming
formulation, thus is guaranteed to converge to a feasible point (if such point exists). However, the formulation is NP
complete in most cases [33], thus convergence to a feasible solution can be very slow. Since both numerical methods
for nonlinear systems and the multicommodity formulation are fundamental questions in their own domains, they
are out of the scope of this paper.
Page 18
18
B. Communication cost
In the case of undirected AVT codes, convergence rates and communication costs for the CAR problem are difficult
to characterize. For this reason, we focus on the partially directed codes and derive corresponding communication
cost. We remind that the partial AVT solutions has two time scales in terms of convergence: 1) The time it takes
for each source class converge to the average, 2) The finite time that it takes for the directed flow from sources to
destination to converge.
A 2-D geometric random graph G2(N,r) consists of N nodes which are uniformly distributed on a unit torus, and
two nodes i and j are said to be connected when d(i,j) ≤ r, where d(.,.) denotes the Euclidean distance of its
arguments. 2-D random geometric graphs are of particular interest since they have been widely used as simplified
models for wireless network topologies and focusing on these graphs makes it possible to compare our algorithm
with regular average consensus algorithms in terms of communication cost [35], [36].
For r = w1(?log(N)/N), it has been shown that the graph is connected with high probability2(w.h.p.) [35], thus
we focus on this regime. Moreover, in that particular regime, the underlying graph is regular with w.h.p., i.e., each
node has Θ(Nr2) neighbors [25], [36]. Therefore, the diameter of the underlying network, i.e., the longest shortest
path, can be bounded as Θ(r−2).
We will assume that each source class utilizes the average consensus protocol with constant edge weights [5]. For
a given random graph with N nodes and radius r = w(?log(N)/N), the communication cost of the average
consensus protocol is Θ(N r−2) [5], [36]. In that particular setting, the communication cost is NTc, where Tcis
the convergence time of the algorithm and it is defined as:
Tc= −
?
log sup
x(0)?=¯ x
lim
t→∞
?||x(t) − ¯ x||2
||x(0) − ¯ x||2
??−1
,
where ¯ x =
1
N
?N
i=1xi(0). In words, the convergence time Tcgives the (asymptotic) number of steps for the error
to decrease by the factor 1/e, and NTcis the total number of message exchanges required for that to happen.
Similarly, for a given CAR problem with source set SS, and corresponding source classes SC =?SSC1,SSC2,...,S|SC|
communication complexity for all classes to converge to their own averages (by 1/e) is equal to:
?,
C = Θ
|SC|
?
i=1
|SSCi| r−2
= Θ?M r−2?,
where each class runs the average consensus algorithm independently. Moreover, once the source classes converge
to the average, the algorithm needs, O(r−2) more steps for each class to distribute the source information to each
1w(.) denotes asymptotic dominatination, Θ(.) denotes asymptotic lower and upper bound and, O(.) denotes asymptotic upper bound,
where the asymptotics are with respect to N.
2with probability at least 1 − 1/N2
Page 19
19
destination. We note that Θ(r−2) is equal to the diameter of the underlying network. Therefore, the algorithm
needs, O(|SC|Kr−2) message exchanges to distribute the information to the destination. We note that as |SC| and
K increases, the paths between source-destination pairs will intersect more and more, thus the bound will become
looser.
The communication cost of the algorithm is equal to:
CAV T= Θ(M r−2) + O(|SC|K r−2) = O([M + |SC|K] r−2)
(17)
Please note that, in our case, the convergence time is defined as:
TAV T= −
?
log sup
xD(0)?=¯ xS
lim
t→∞
?||xD(t) − ¯ xS||2
||xD(0) − ¯ xS||2
??−1
,
where xSand xDare the states of source and destination nodes respectively.
If we were to solve our problem via regular average consensus algorithm as we have discussed in Section II, the
communication cost is Θ(N r−2). When, the number of sources, source classes and destination nodes are small,
i.e., M,SC,K ? N, the bound in (17) is smaller than O(N r−2). We note that this is the regime where (17)
is tight, thus, we expect that partial AVT solutions will be N/[M + |SC|K] more efficient than regular average
consensus solutions. In the case that SC,K ? N, our bound is still tight, and AVT will be N/M more efficient.
For the regimes where |SC|,K are large, our bound in (17) becomes loose. While it is true that the complexity of
the partial AVT solutions will increase, the true complexity is expected to be sub-linear in |SC| and K.
VIII. SIMULATIONS
In this section, we will simulate the behavior of the directed AVT solutions on random graphs, and compare
their performances with regular average consensus solutions. Due to the construction complexity of the undirected
AVT formulations, they will not be included in our simulations. We first simulate the communication complexity
of the directed AVT solutions with respect to the number of source nodes M. To keep the simulation setup as
simple as possible, we fix the number of source classes and destinations nodes as one. We choose N = 100
and r =
?2log(N)/N. For each value of M, we determine a connected subgraph of the network, and run the
average consensus algorithm with constant edge weights [5]. We determine the communication cost as the number
of messages required for the error to decrease by the factor 1/e. We then calculate the number of hops from
the source class to the destination node, and add this value to determine the total cost. We generate 100 random
geometric graphs for each M, and also averaged our results over 100 random initial node values. Moreover, for
each of these initial conditions, we simulate the communication cost of the regular average consensus algorithm,
i.e., nodes calculate the average of the whole network. In Fig.6(a), we plot the ratio of the communication costs
Page 20
20
20
Number of Source Nodes
4060 80
1
2
3
4
5
Ratio of Communication Costs
(a) Ratio of the communication costs of ACG and directed AVT.
05 1015
0
10
20
30
40
Number of Source Clusters
Communication Complexity
(b) The communication cost of directed AVT.
Fig. 6.
of the regular average consensus and the directed AVT algorithms. As we have mentioned in Section VII, AVT
The communication costs of directed AVT.
requires M/N < 1 communication exchanges with respect to the regular average consensus algorithm. The regime
is linear with respect to M when M ≤ N/2, and becomes logarithmic for M = Θ(N).
In the second part, we simulate the communication complexity of the directed AVT solutions with respect to the
number of source classes |SC|. We once again fix the number of destinations as one. We choose N = 100 and
r =
?2log(N)/N. For each value of |SC|, we pick |SC| isolated nodes (in the sense that none of them are
neighbors). We then solve the IP programming formulation given in Section VI. We, once again, average our
results over 100 random graphs. In Fig. 6(b), we plot the communication complexity versus the number of source
classes. As we have discussed in Section VII, the cost increase linearly as the number of source classes increases
for |SC| ? N.
IX. CONCLUSION
In this paper, we studied the distributed computation problem, where a set of destination nodes are interested in the
average of another set of source nodes. Utilizing gossiping protocols for the information exchange, we provided
necessary and sufficient conditions on the feasibility of AVT codes. Moreover, we showed that the feasibility of the
problem only depends on the asymptotic behavior of the source states and is independent of the evolution of these
states. We analyzed the spectral properties of feasible codes, and proposed several simplifications which reduce
the complexity of the problem without affecting the feasibility. By focusing on stochastic updates, we provided
necessary condition on the feasibility that are easy to interpret, and then discussed some known infeasible scenarios.
Since the feasible region of AVT codes are non-convex, we introduce so called partially directed AVT solutions,
and provided integer programming formulation for constructing such solutions. We analyzed the complexity and
communication cost of the algorithm and compared the performance of our algorithm with existing solutions.
Page 21
21
APPENDIX A
1) We first note that W∞is an idempotent matrix. Thus by using (3) and (4):
W∞W∞= W∞⇒ B?D?= E?= 0.
Since B?= 1/|SS|11T> 0 and D?≥ 0, the above equality holds if and only if D?= 0. The proof that
G?= 0 follows easily and therefore it is omitted.
2) We first note that by the first part of the lemma, W∞has the structure:
W∞=
A?
00
1/|SS|11T
00
C?
F?
P?
.
Since W∞W = W∞, then 1/|SS|11TD = 0. Since D is nonnegative, then D = 0. Similarly, 1/|SS|11TG =
0 and G ≥ 0, thus G = 0.
3) By noting once again W∞W = W∞, 1/|SS|11TA = 1/|SS|11T. Therefore, 1TA = 1T.
APPENDIX B
We first prove that if limt→∞
??t
l=0Dt−lBAl?
1:K=
1
M11T, then limt→∞
??t
l=0Dt−lBA∞?
1:K=
1
M11T. Let’s
denote by B?= limt→∞
?t
l=0Dt−lBAl, and recall that B?< ∞ exists by W∞< ∞ assumption. If we multiply
both sides of the equality by A∞
B?A∞=
?
?
lim
t→∞
t
?
t
?
l=0
Dt−lBAl
?
A∞
=lim
t→∞
l=0
Dt−lBAlA∞
?
= lim
t→∞
t
?
l=0
Dt−lBA∞,
where the second equality follows from the fact that A∞is a fixed matrix and B?< ∞ (therefore one can
take A∞inside the limit), and the third equality is due to the fact that AlA∞= A∞∀l ≥ 0. Since the
column sum of A is constant and equal to 1 (by Lemma 1) and A is a non-negative matrix, then λ = 1
is an eigenvalue of A corresponding to eigenvector 1T[37]. Therefore, 1TA∞= 1T. Keeping that in mind,
[B?A∞]1:K= 1/|SS|11TA∞= 1/|SS|11T= [B?]1:K. This concludes the first part of the proof.
Now, we show that, if limt→∞
??t
l=0Dt−lBA∞?
1:K=
1
M11T, then limt→∞
??t
l=0Dt−lBAl?
1:K=
1
M11T. Let’s
denote B?= limt→∞
?t
l=0Dt−lBA∞, and B?< ∞ exists by assumption. Then,
B?= lim
t→∞
t
?
l=0
Dt−lBA∞
Page 22
22
=
?
lim
t→∞
t
?
l=0
Dt−lBAlA∞
?
=
?
lim
t→∞
t
?
l=0
Dt−lBAl
?
A∞,
where the second equality follows from the fact that AlA∞= A∞. To prove the third equality, we first note that
limt→∞
?t
l=0Dt−lBAlexists (although its value may not be finite) since, each term in the expression is non-
negative. Since A∞does not have all zeros rows, each term in the matrix
?limt→∞
?t
l=0Dt−lBAl?
is present
in B?with a strictly positive coefficient. Therefore, one may take A∞outside of the limit. We also note that
nonexistence of all zeros row in A∞matrix can be shown by considering the equality AA∞= A∞.
Let’s define L?=
?limt→∞
?t
l=0Dt−lBAl?. Since W∞W∞= W∞, then L?A∞+ D?L?= L?. Noting that
L?A∞= B?and the first K rows D?matrix are all zeros by the third condition of Theorem 1, the first K rows of
L?is equal to the first K rows of B?. Therefore, limt→∞
??t
l=0Dt−lBAl?
1:K=
1
M11T. This concludes the proof.
APPENDIX C
We first note that the rank of a limiting matrix W∞is equal to both the number of eigenvalues of W that are equal
to 1 and the number of eigenvalues of W∞that are equal to 1. Moreover, eigenvalues of W∞are the union of the
eigenvalues of A∞and D∞since W∞is in the block lower triangular form [38]. Since any given source class
SCiforms an irreducible sub-network, the largest eigenvalue of ASCiis equal to 1 (since column sum is equal to
1 by Lemma 1) and it is unique in magnitude (due to irreducibility) [38]. Such an argument holds for all source
classes. Thus, the number of non-zero eigenvalues of A∞= limt→∞Atis equal to the number of source classes.
Therefore, this is a lower bound for the rank of W∞.
To prove the upper bound, we first note that we can partition the network into classes. Moreover, these classes are
irreducible and disjoint by the definition. For a given class, the spectral radius upper bounded by 1 and the largest
eigenvalue is unique and equal to the spectral radius [38]. Therefore, W∞can have at most # of classes non-zero
eigenvalues.
APPENDIX D
Given ρ(D) < 1, limt→∞Dt= 0 [37], and the fourth condition in the Theorem 1 simplifies to:
lim
t→∞
t
?
l=0
?
[Dl]1:KBA∞?
= lim
t→∞
?
t
?
l=0
[Dl]1:K
?
BA∞
= [(I − D)−1]1:KBA∞=
1
M11T,
(18)
Page 23
23
S1S2
S3S4
S5
Fig. 7.
where the third equality follows from the fact that limt→∞
Source clustering due to W?. Dashed line between S2 and S3 represents zero weight link. Cloud represents rest of the network.
?t
can be rewritten as:
l=0[Dl] = [(I − D)−1] when ρ(D) < 1 [38]. (18)
R(I − D)−1BA∞=
1
M11T,
(19)
where R ∈ RK×Nwith diagonal elements equal to one, and the rest are all equal to zero. Since the rank of
1/M11Tis one, then the rank of the multiplication on the left hand side of the equality has to be equal to one. At
this point, we remind our readers the following well-known result regarding the rank of a matrix multiplication:
Lemma 9. [39] If T is m × n and Q is n × p, then rank(TQ) ≥ rank(T) + rank(Q) − n.
By noting that the ranks of R,(I −D)−1,A∞,11Tare equal to K, N, # of source classes and 1 respectively, our
result follows from (19).
APPENDIX E
Consider CAR F(SS,SD) and a network F(E). By hypothesis, the problem has at least one feasible solution,
namely W?. If W?utilizes the minimal source class, then the lemma is satisfied. Let us assume that W?does
not utilize the minimal class, i.e., it assigns zero weights to some of the existing links in between source nodes.
Without loss of generality, we assume that under minimal class the network has only one class, and W?has two
source classes since it assigns zero weight to a link between a pair of source nodes. An example network is given
in Fig. 7. We further assume that the source nodes have already converged to their limiting states. We note that
this assumption does not change the feasibility of the problem due to Lemma 2. Without loss of generality, we
also assume that each source’s limiting state value is the average of the initial values of the source nodes in that
particular class.
Let there be M1source nodes in Cluster 1 and M2source nodes in Cluster 2 and M1+ M2= M. Source nodes
in class 1 and 2 have already converged to 1/M1
?
i∈C1xi(0) and 1/M2
?
i∈C2xi(0) respectively, where Ciis the
set of nodes in class i. Since W?is a feasible code, all of the destinations in the network converge to the average
Page 24
24
of the all source nodes. But, this is only possible if destination nodes have access to:
M1
M
?
1
M1
?
i∈C1
xi(0)
?
+M2
M
?
1
M2
?
i∈C2
xi(0)
?
=
1
M
?
i∈SS
xi(0),
(20)
in the limit. At this point, we design the minimal class code¯ W from W?as follows: We assign non-zero weights
to all of the links between source nodes, and choose these weights such that source nodes converge to the average
of the initial source values. Since all of the links in between source nodes are assigned non-zero weight, ¯ W
corresponds to the minimal class. We keep B and D partitions of W?unchanged, utilize these in¯ W. Since, the
link weights governing the communication among non-source nodes (partition D) and also between sources and
non-source nodes (partition B) have not changed, the destination nodes will converge to:
M1
M
?
1
M
?
i∈SS
xi(0)
?
+M2
M
?
1
M
?
i∈SS
xi(0)
?
=
1
M
?
i∈SS
xi(0).
(21)
We note that the equation given above is very similar to (20). In particular, the coefficients in front of the first and
the second terms in the summations are exactly the same. As we have mentioned above, this is true since B and
D partitions of W?remains unchanged. Moreover, the terms in the parenthesis have changed, since each source
node now has access to the global average.
Since (21) is equal to the average the source nodes,¯ W is a feasible minimal class solutions and hence the lemma
is proved.
The proof can be easily generalized to the case of several source classes and to the case where all sources do not
converge to the average of the initial source node values in that particular class, by accounting more than two terms
in (20)-(21).
APPENDIX F
Consider a network F(E) and the source and destinations sets, SSand SD, respectively. We partition the network
into two disjoint sets (P,Pc) such that there exists at least one source class-destination pair on both sides of the
network. We consider the case where there exists at least one partition such that the number of cut edges is strictly
less than two. The graphical representation of the network is given in Fig. 8. The curve represents the boundary
between two disjoint partitions. SS1 represents all of the sources nodes in partition P, and D1 represents all of the
destination nodes in partition P. Similarly, SS2 and D2 represents the source and destination nodes in partition
Pc. We note that the number of cut edges cannot be zero since otherwise the network would be disconnected. Thus
the number of cut edges between P and Pchas to be equal to one. We will assume that there exists a feasible W
for this problem, and prove our claim by finding a contradiction.
In the network topology given in Fig. 8, we denote I1 and I2 as bottleneck nodes, i.e., the edge between I1 and
Page 25
25
SS1
I1I2
D1
SS2
D2
Fig. 8.
I2 is the only cut edge for the partition. We index the source nodes in the network such that the first M1source
A network with two partitions and a single cut edge.
nodes are in P and the last M2source nodes are in Pc. We note that M1+ M2= M. We refer to W∞
ijas the
weight of the node j at node i.
Since, there is only one cut edge between partition P and Pc, information from SS2 to D1 has to flow through the
edge (I2, I1). As we have discussed in Section V, W matrix is indeed a transition probability matrix for the chain
M, W∞consists of corresponding absorbtion probabilities by the source classes. It is clear that SS1 and SS2
are absorbing classes in our case. The probability of being absorbed by SS2 when the chain has been initialized
at state I2 is larger than the probability of being absorbed by SS2 when the chain has been initialized at state
I1, since, in the latter case, the chain has to visit I2 before being absorbed by SS2. Therefore, for all k ∈ SS2,
the frequency of visiting k in the long run starting from I2 is larger than the frequency of visiting k in the long
run starting from I1. Since these probabilities are measured by the entries of W∞matrix, the argument follows.
Mathematically speaking, our result is:
W∞
I1,k≤ W∞
I2,k
∀ k ∈ {M1+ 1,...,M}.
(22)
Similarly, the weight of SS1 at I1 has to be greater than or equal to the weight of SS1 at I2, i.e.,
W∞
I2,k≤ W∞
I1,k
∀ k ∈ {1,...,M1}.
(23)
Similarly, the weight of SS2 at D1 is less than or equal to the weight of SS2 at I1 and, the weight of the SS1
information to D2 is less than or equal to that of SS1 at I2. Moreover, since, W is feasible, sources’ weights at
destination nodes D1 and D2 are equal to 1/M. Combining these facts with (22)-(23):
1
M≤ W∞
I1,k≤ W∞
I2,k
∀ k ∈ {1,...,M1},
(24)
1
M≤ W∞
I2,k≤ W∞
I1,k
∀ k ∈ {M1+ 1,...,M}.
(25)
Page 26
26
Since W1 = 1, then W∞1 = 1:
M
?
k=1
W∞
I1,k= 1 and
M
?
k=1
W∞
I2,k= 1.
(26)
Since all of the terms in both summations are lower bounded by 1/M, and there are M terms in both summations,
the equality in (26) holds if and only if W∞
I1,k= W∞
I2,k= 1/M ∀k ∈ {1,...M}. Such an equality implies that
I1 and I2 also converge to the average.
This observation is particularly important for the following reason: Since we are focusing on the limiting values of
the node states, they are invariant with respect to the transformation with respect to W matrix, i.e., WW∞= W∞.
In particular, the weight of SS2 at node I1 has to be equal to a convex combination of the weights of SS2 at
node I1 and at I1’s neighbors. We have already shown that the weight of SS2 at I1 and I2 (which is a neighbor
of I1) are equal to 1/M. Since all the neighbors of I1 except I2 belong to the partition P, the weight of SS2 at
each of these nodes is upper bounded by 1/M. Then the weight of SS2 at all of the neighbors of I1 has to be
equal to the 1/M. Otherwise, this would not be a stable point.
One can now focus on the neighbor set of node I1 (denoted by NI1) and argue that the neighbors of NI1also
have 1/M as the weight of node SS2 since all of them belong to the partition P. One can utilize the argument
above iteratively to show that the weight of SS2 at all of the nodes in P partition converge to 1/M.
However, this is a contradiction since the weight of SS2 at SS1 cannot be equal to 1/M, since since these nodes
can not communicate with SS2. Therefore, there does not exist any feasible code for such a topology.
APPENDIX G
We first to note that the integer programming formulation given in the lemma is a directed multicommodity flow
problem with acyclicity constraint. In particular, one can map the variable bkl
ito net inflow at node i of data with
origin k and destination l. zkl
ijindicates the amount of information with origin k and destination l that flows through
link (i,j). yk
ijis equal to one if there exists at least one flow on (i,j) that is originated from source class k.
We assume that the integer programming formulation has a feasible solution. We will show that W, which is
constructed as in (16), is a feasible AVT solution.
We first note that for a given source node i, we assign equal weights to all of its source neighbors and itself.
Therefore, each source node in a given class will converge to the average of the nodes in that particular class (c.f.
[5]). Utilizing Lemma 2, we can assume that source nodes have already converged to the averages.
Moreover, W in (16) is stochastic by construction, therefore it has a limit. By Theorem 3.1 of [40], the rank of
W∞is equal to the number of source classes in the network. Similar to our discussion in Section IV-A, we denote
the columns of W∞as um,1 ≤ m ≤ rW∞. These columns are the eigenvectors of W corresponding to the
Page 27
27
eigenvalue 1.
We will prove that m-th eigenvector umwill have the following structure:
[um]j=
1
|SCm|,
if j ∈ SCm,
0,
if j ∈ SSand j ?∈ SCm,
1
?
l∈Nj
?
k∈SC|SC(k)|yk
lj,
if?
l∈Njym
lj?= 0,
0,
otherwise,
(27)
where SCmis the set of source nodes that belong to the source class m and [.]j denotes the j-th element of its
argument. If umis an eigenvector of W corresponding to the eigenvalue 1, then Wum= ummust hold, i.e.:
[Wum]j=
N
?
i=1
Wji[um]i ∀j ∈ {1,2,...,N}.
If node j is a source node which is in the class m, then:
[Wum]j=
N
?
i=1
Wji[um]i=
1
|SCm|
?
i∈{Nj∪j}
1
|Nj| + 1=
1
|SCm|,
where the second equality follows from the fact that all of the neighbors of node j belongs to the set SCm, and the
third equality is due to the fact that each of the incoming links has the same weight, i.e. 1/{|Nj| + 1}, by (16).
Therefore, [Wum]j= [um]j, ∀j ∈ SCm.
If j is a source node, but does not belong to the class m, then:
[Wum]j=
N
?
i=1
Wji[um]i=
?
i∈SS\SCm
Wji[um]j+
?
i∈S\(SS\SCm)
Wji[um]j=
?
i∈SS\SCm
Wji0 +
?
i∈S\(SS\SCm)
0[um]j= 0 = [u]j,
where \ denotes the ”set difference”. The third equality follows from the construction of umin (27) and the fact
that source nodes, which do not belong to the source class m, cannot hear from the class m.
If j is not a source, then:
[Wum]j=
N
?
i=1
Wji[um]i=
?
i∈SCm
Wji[um]i+
?
i∈{SS\SCm}
Wji[um]i+
?
i?∈SS
Wji[um]i.
(28)
We note that the first term above represents the summation over the source nodes which are in class m, the second
term is the summation over the rest of the source nodes, and the last summation is over the non-source nodes. The
second summation in (28) is zero, since [um]i= 0 for all source nodes i which are not included in the source
class m by the construction in (27). Moreover, of all the terms in the first and the third summation, only a single
term can be non-zero. This is true due to the acyclicity constraint in (14). If there were two non-zero terms in the
summation, this would mean that node j has two neighbors both of which carry information from the source class
Page 28
28
m. But, this would result in a cycle in the flow graph. Thus, such flow would not be a feasible solution to the
integer programming formulation given in the lemma.
We first assume that there exists a non-zero term, the term is in the first summation in (28) and, we denote the
index of this term as i. Then by (16) and (27):
[Wum]j=
|SCm|yijm
?
?
l∈Nj
k∈SC|SC(k)|yk
lj
1
|SCm|=
1
?
l∈Nj
?
k∈SC|SC(k)|yk
lj
= [um]j.
Note that ym
ij= 1, since we assumed that there is a flow from node i to node j which is originated from class m.
Next, we assume that the non-zero term is in the third summation in (28) and denote the index of this term as i.
Then by (16) and (27):
[Wum]j=
?
l∈Nj
k∈SC|SC(k)|yk
?
ij
?
k∈SC|SC(k)|yk
lj
1
?
l∈Ni
?
k∈SC|SC(k)|yk
li
.
(29)
We remind the reader that if the term above is non-zero, then?
One can think of?
corresponding class sizes) that are going into the node i. Moreover, if there exists at least one flow from node i to
l∈Ni
?
k∈SC|SC(k)|yk
li=?
k∈SC|SC(k)|yk
ij.
l∈Ni
?
k∈SC|SC(k)|yk
lias the total number of flows from distinct classes (rescaled by the
node j, then there has to be at least one flow from node i to node j from each distinct class whose information is
present at the node i. Otherwise, acyclicity constraint would not hold. Therefore, the equality holds. Then, we can
rewrite (29) as:
[Wum]j=
1
?
l∈Nj
?
k∈SC|SC(k)|yk
lj
= [um]j.
Now, we assume that all of the terms in (28) are zero. But due to the construction of W and um, this is only
possible if there is no flow that is going into node j from source class m. If this is the case, [Wum]j= [um]j= 0.
Since we covered all possible cases, this proves that {um}rW∞
m=1are the eigenvectors of W corresponding to the
eigenvalue 1. Moreover, due to their constructions, these eigenvectors are linearly independent.
We construct the left eigenvectors corresponding to the eigenvalue 1 as follows:
[cm]j=
1,
if j ∈ SCm,
0,
otherwise.
Moreover, it is easy to check that cT
mun= 0 if m ?= n. Then by [41]:
lim
t→∞Wt=
rW∞
?
m=1
umcT
cT
mum
m
=
rW∞
?
m=1
umcT
m,
where the second equality follows form the fact that cT
mum= |SCm|/|SCm| = 1 forall m. At this point, we need
to observe that [um]j = 1/|SS| if j ∈ SD, due to the integer programming formulation in the lemma and the
Page 29
29
eigenvector construction in (27). This concludes the proof.
ACKNOWLEDGMENT
We would like to thank Dr. Tuncer C. Aysal, and anonymous reviewers for their insightful comments and sugges-
tions.
REFERENCES
[1] M. Yildiz, T. C. Aysal, and A. Scaglione, “Network information flow: Gossiping with groups,” IEEE Asilomar, 2008.
[2] M. E. Yildiz, A. Scaglione, and T. C. Aysal, “Computing along routes via gossiping,” in Information Theory Workshop, October 2009.
[3] M. E. Yildiz and A. Scaglione, “Directed gossiping for distributed data aggregation,” in IEEE CAMSAP, December 2009.
[4] R. Ahlswede, N. Cai, S.-Y. R. Li, and R. W. Yeung, “Network information flow,” IEEE Transactions on Information Theory, pp.
1204–1216, 2000.
[5] L. Xiao and S. Boyd, “Fast linear iterations for distributed averaging,” Systems and Control Letters, vol. 53, pp. 65–78, 2004.
[6] R. Koetter and M. Medard, “An algebraic approach to network coding,” IEEE/ACM Transactions on Networking, vol. 11, pp. 782–795,
2003.
[7] B. Krishnamachari, D. Estrin, and S. B. Wicker, “The impact of data aggregation in wireless sensor networks,” in Proceedings of the
22nd International Conference on Distributed Computing Systems, Washington, DC, USA, 2002, pp. 575–578.
[8] F. Zhao, J. Liu, J. Liu, L. Guibas, and J. Reich, “Collaborative signal and information processing: An information directed approach,”
in Proceedings of the IEEE, 2003.
[9] S. Madden, R. Szewczyk, M. J. Franklin, and D. Culler, “Supporting aggregate queries over ad-hoc wireless sensor networks,” in IEEE
Workshop on Mobile Computing and Systems Applications, 2002, pp. 49–58.
[10] R. Olfati-Saber and R. M. Murray, “Consensus problems in networks of agents with switching topology and time-delays,” IEEE
Transactions on Automatic Control, vol. 49, no. 9, pp. 1520–1533, 2004.
[11] M. Ji, A. Muhammad, and M. Egerstedt, “Leader-based multi-agent coordination: controllability and optimal control,” in American
Control Conference, 2006, June 2006.
[12] E. W. Dijkstra, “A note on two problems in connexion with graphs,” Numerische Mathematik, pp. 269–271, 1959.
[13] S. E. Deering, “Multicast routing in a datagram internetwork,” Ph.D. dissertation, Stanford University, 1992.
[14] R. Dougherty, C. Freiling, and K. Zeger, “Insufficiency of linear coding in network information flow,” IEEE Transactions on Information
Theory, pp. 2745–2759, 2005.
[15] R. Koetter and M. Medard, “An algebraic approach to network coding,” IEEE Transaction on Networking, pp. 782–795, 2003.
[16] M. Enachescu, A. Goel, R. Govindan, and R. Motwani, “Scale-free aggregation in sensor networks,” Theor. Comput. Sci., pp. 15–29,
2005.
[17] S. Nath, P. B. Gibbons, S. Seshan, and Z. R. Anderson, “Synopsis diffusion for robust aggregation in sensor networks,” in Proceedings
of the 2nd international conference on Embedded networked sensor systems, 2004, pp. 250–262.
[18] P. Ciciriello, L. Mottola, and G. P. Picco, “Efficient routing from multiple sources to multiple sinks in wireless sensor networks,”
Lecture Notes in Computer Science, vol. 4373, April 2007.
[19] J. N. Al-Karaki, R. Ul-Mustafa, and A. E. Kamal, “Data aggregation and routing in wireless sensor networks: Optimal and heuristic
algorithms,” Computer Networks, vol. 53, no. 7, pp. 945–960, 2009.
Page 30
30
[20] S. Madden, M. J. Franklin, J. M. Hellerstein, and W. Hong, “The design of an acquisitional query processor for sensor networks,” in
ACM SIGMOD, 2003, pp. 491–502.
[21] Y. Yao and J. Gehrke, “Query processing for sensor networks,” Cornell University, 2003.
[22] C. Intanagonwiwat, R. Govindan, D. Estrin, J. Heidemann, and F. Silva, “Directed diffusion for wireless sensor networking,” IEEE/ACM
Trans. Netw., vol. 11, no. 1, pp. 2–16, 2003.
[23] T. C. Aysal, M. E. Yildiz, A. D. Sarwate, and A. Scaglione, “Broadcast gossip algorithms for consensus,” IEEE Trans on Signal
Processing, Accepted, December 2008.
[24] M. E. Yildiz and A. Scaglione, “Coding With Side Information for Rate-Constrained Consensus,” IEEE Transactions on Signal
Processing, vol. 56, pp. 3753–3764, Aug. 2008.
[25] F. Benezit, A. Dimakis, P. Thiran, and M. Vetterli, “Gossip along the way: order-optimal consensus through randomized path averaging,”
in Allerton, 2007.
[26] U. A. Khan, S. Kar, and J. M. F. Moura, “Distributed sensor localization in random environments using minimal number of anchor
nodes,” IEEE Transactions on Signal Processing,Accepted, 2008.
[27] D. Mosk-Aoyama and D. Shah, “Information dissemination via network coding,” in Information Theory, 2006 IEEE International
Symposium on, July 2006, pp. 1748–1752.
[28] ——, “Fast distributed algorithms for computing separable functions,” Information Theory, IEEE Transactions on, vol. 54, no. 7, pp.
2997–3007, July 2008.
[29] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Randomized gossip algorithms,” IEEE Trans. Info. Theory, vol. 52, no. 6, pp.
2508–2530, June 2006.
[30] J. Tsitsiklis, “Problems in decentralized decision making and computation,” Ph.D. dissertation, Dept. of Electrical Engineering and
Computer Science, M.I.T., Boston, MA, 1984.
[31] T. Aysal, M. Coates, and M. Rabbat, “Probabilistically quantized distributed averaging,” IEEE Transactions Signal Procesing, October
2008.
[32] D. Aldous and J. Fill, “Reversible markov chains and random walks on graphs,” http://www.stat.berkeley.edu/ aldous/RWG/book.html.
[33] D. Bertsimas and J. N. Tsitsiklis, Introduction to Linear Optimization.Athena Scientific, 1997.
[34] J. E. Dennis, Jr. and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Classics in Applied
Mathematics, 16). Soc for Industrial & Applied Math, 1996.
[35] P. Gupta and P. R. Kumar, “The capacity of wireless networks,” IEEE Transactions on Information Theory, vol. 46, pp. 388–404,
March 2000.
[36] S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, “Mixing times for random walks on geometric random graphs,” in Proceedings of the
Workshop on Analytic Algorithms and Combinatorics, Vancouver, BC, Jan. 2005.
[37] F. Gantmacher, Matrix Theory.Chelsea Publishing, 1959, vol. 2.
[38] R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge University Press, Cambridge, 1987.
[39] C. Meyer, Matrix Analysis and Applied Linear Algebra.SIAM: Society for Industrial and Applied Mathematics, 2001.
[40] U. G. Rothblum, “Algebraic eigenspaces of nonnegative matrices,” Linear Algebra and Its Applications, vol. 12, pp. 281–292, 1975.
[41] R. Oldenburger, “Infinite powers of matrices and characteristic roots,” Duke Mathematics Journal, vol. 12, pp. 357–361, 1940.
Download full-text