A New Upper Bound on the Average Error Exponent for Multiple-Access Channels

Abstract

A new lower bound for the average probability or error for a two-user discrete memoryless (DM) multiple-access channel (MAC) is derived. This bound has a structure very similar to the well-known sphere packing packing bound derived by Haroutunian. However, since explicitly imposes independence of the users' input distributions (conditioned on the time-sharing auxiliary variable) results in a tighter sphere-packing exponent in comparison to Haroutunian's. Also, the relationship between average and maximal error probabilities is studied. Finally, by using a known sphere packing bound on the maximal probability of error, a lower bound on the average error probability is derived.

Full text

Page 1

arXiv:1010.1322v1 [cs.IT] 7 Oct 2010
1
A New Upper Bound on the Average Error
Exponent for Multiple-Access Channels
Ali Nazari, S. Sandeep Pradhan and Achilleas Anastasopoulos
Electrical Engineering and Computer Science Dept.
University of Michigan, Ann Arbor, MI 48109-2122, USA
E-mail: {anazari,pradhanv,anastas}@umich.edu
Abstract
A new lower bound for the average probability or error for a two-user discrete memoryless (DM)
multiple-access channel (MAC) is derived. This bound has a structure very similar to the well-known
sphere packing packing bound derived by Haroutunian. However, since explicitly imposes independence
of the users’ input distributions (conditioned on the time-sharing auxiliary variable) results in a tighter
sphere-packing exponent in comparison to Haroutunian’s. Also, the relationship between average and
maximal error probabilities is studied. Finally, by using a known sphere packing bound on the maximal
probability of error, a lower bound on the average error probability is derived.
Index Terms
Multiple-access channel, error exponents, Sphere Packing bound.
I. INTRODUCTION
One of the most important practical questions which arises when we are designing or using an
information transmission or processing system is: How much information can this system transmit or
process in a given time? Information theory, developed by Claude E. Shannon during World War II,
defines the notion of channel capacity and provides a mathematical model by which one can compute
it. Basically, Shannon coding theorem and all newer versions of it treat the question of how much data
can be reliably communicated from one point, or sets of points, to another point or sets of points.
The class of channels to be considered include multiple transmitter and a single receiver. The received
signal is corrupted both by noise and by mutual interference between the transmitters. Each of trans-
mitters is fed by an information source, and each information source generates a sequence of messages.
More specifically, a two-user DM-MAC is defined by a stochastic matrix1W : X × Y → Z, where the
input alphabets, X, Y, and the output alphabet, Z, are finite sets. The channel transition probability for
sequences of length n is given by
Wn(z|x,y) ?
n
?
i=1
W (zi|xi,yi)
(1)
This work was supported by NSF ITR grant CCF-0427385.
1We use the following notation throughout this work. Script capitals U, X, Y, Z,... denote finite, nonempty sets. To show the
cardinality of a set X, we use |X|. We also use the letters P, Q,... for probability distributions on finite sets, and U, X, Y ,... for
random variables.
October 8, 2010 DRAFT

Page 2

2
where
x ? (x1,...,xn) ∈ Xn,y ? (y1,...,yn) ∈ Yn
and
z ? (z1,...,zn) ∈ Zn.
It has been proven, by Ahlswede [1] and Liao’s [6] coding theorem, that for any (RX,RY) in the interior
of a certain set C, and for all sufficiently large n, there exists a multiuser code with an arbitrary small
average probability of error. Conversely, for any (RX,RY) outside of C, the average probability of error
is bounded away from 0. The set C, called capacity region for W, is the closure of the set of all rate pairs
(RX,RY) satisfying [12]
0 ≤ RX≤ I (X ∧ Z|Y,U)
(2a)
0 ≤ RY ≤ I (Y ∧ Z|X,U)
(2b)
0 ≤ RX+ RY ≤ I (XY ∧ Z|U),
(2c)
for all choices of joint distributions over the random variables U, X, Y, Z of the form p(u)p(x|u)p(y|u)W (z|x,y)
with U ∈ U and |U| ≤ 4. As we can see, this theorem was presented in an asymptotic nature, i.e., it was
proven that the error probability of the channel code can go to zero as the block length goes to infinity. It
does not tell us how large the block length must be in order to achieve a specific error probability. On the
other hand, in practical situations, there are limitations on the delay of the communication. Additionally,
the block length of the code cannot go to infinity. Therefore, it is important to study how the probability
of error drops as the block length goes to infinity. A partial answer to this question is provided by
examining the error exponent of the channel.
Error exponents have been studied for discrete memoryless multiple-access channels over the past
thirty years. Lower and upper bounds are known on the error exponent of these channels. The random
coding bound in information theory provides a well-known lower bound for the reliability function of
the best code, of a given rate and block length. This bound is constructed by upper-bounding the average
error probability over an ensemble of codes. Slepian and Wolf [12], Dyachkov [3], Gallager [4], Pokorny
and Wallmeier [11], and Liu and Hughes [7] have all studied the random coding bound for discrete
memoryless multiple access channels. Nazari and et al. [8] investigated two different upper bounds on the
average probability of error, called the typical random coding bound and the partial expurgated bound.
The typical bound is basically the typical performance of the ensemble. By this, we mean that almost
all random codes exhibit this performance. In addition, they have shown that the typical random code
performs better than the average performance over the random coding ensemble, at least, at low rates.
The random coding exponent may be improved at low rates by a process called “partial expurgation”
which yields a new bound that exceeds the random coding bound at low rates.
Haroutunian [5] and Nazari [9], [10] studied upper bounds on the error exponent of multiple access
channels. In Multi-user information theory, the sphere packing bound provides a well known upper
bound on the reliability function for multiple access channel. The sphere packing bound that Haroutu-
nian [5] derived on the average error exponent for DM-MAC is potentially loose, as it does not capture
the separation of the encoders in the MAC. Nazari et al. [10] derived another sphere packing bound
which takes into account separation of the encoders. The bound in [10] turns out to be at least as good as
October 8, 2010DRAFT

Page 3

3
the bound derived in [5], however it is a valid bound only for the maximal error exponent and not the
average. The sphere packing bound is a good bound in high rate regime. Nevertheless, it tends to be a
loose bound in low rate regime. It can be shown that in low rate regime, the minimum distance of the code
dominates the probability of error. Using the minimum distance of the code, Nazari [9] derived another
upper bound for the maximal error exponent of DM-MAC. To derive the minimum distance bound, they
established a connection between the minimum distance of the code and the maximum probability of
error; then, by obtaining an upper bound on the minimum distance of all codes with certain rates, they
derived a lower bound on the maximal error probability that can be obtained by a code with a certain
rate pair.
The paper is organized as follows. Some preliminaries are introduced in section II. The main result of
the paper, which is an upper bound on the reliability function of the channel, is obtained in section III.
In section IV, by using a known upper bound on the maximum error exponent function, we derive an
upper bound on the average error exponent function. The proofs of some of these results are given in
the Appendix.
II. PRELIMINARIES
For any alphabet X, P (X) denotes the set of all probability distributions on X. The type of a sequence
x = (x1,...,xn) ∈ Xnis the distributions Px, on X, defined by:
Px(x) ?1
nN (x|x),x ∈ X,
(3)
where N (x|x) denotes the number of occurrences of x in x. Let Pn(X) denotes the set of all types in
Xn, and define the set of all sequences in Xnof type P as
TP? {x ∈ Xn: Px= P}.
(4)
The joint type of a pair (x,y) ∈ Xn× Ynis the probability distribution Px,yon X × Y defined by:
Px,y(x,y) ?1
nN (x,y|x,y),
(x,y) ∈ X × Y,
(5)
where N (x,y|x,y) is the number of occurrences of (x,y) in (x,y). The relative entropy or Kullback-Leibler
distance between two probability distribution P,Q ∈ P (X) is defined as
D(P||Q) ?
?
x∈X
P (x)logP (x)
Q(x).
(6)
Let W (Y|X) denote the set of all stochastic matrices with input alphabet X and output alphabet Y. Then,
given stochastic matrices V, W ∈ W (Y|X), the conditional I-divergence is defined by
D(V ||W|P) ?
?
x∈X
P (x)D(V (·|x)||W (·|x)).
(7)
Definition 1. An (n,M,N) multi-user code is a set {(xi,yj,Dij) : 1 ≤ i ≤ M,1 ≤ j ≤ N} with
• xi∈ Xn, yj∈ Yn, Dij⊂ Zn
• Dij∩ Di′j′ = ∅ for (i,j) ?= (i′,j′).
October 8, 2010DRAFT

Page 4

4
The average error probability of this code for the MAC, W : X × Y → Z, is defined as
e(C,W) ?
1
MN
M
?
i=1
N
?
j=1
Wn?Dc
i,j|xi,yj
?.
(8)
Similarly, the maximal error probability of this code for W is defined as
em(C,W) ? max
(i,j)Wn?Dc
i,j|xi,yj
?.
(9)
Definition 2. For the MAC, W : X × Y → Z, the average and maximal error reliability functions, at rate pair
(RX,RY), are defined as:
E∗
av(RX,RY) ? lim
n→∞max
C
−1
nloge(C,W)
−1
nlogem(C,W),
(10)
E∗
m(RX,RY) ? lim
n→∞max
C
(11)
where the maximum is over all codes of length n and rate pair (RX,RY).
Definition 3. A code CX= {xi∈ Xn: i = 1,...,MX}, for some PX, is called a bad codebook, if
∃ (i,j),i ?= j
xi= xj
(12)
A codebook which is not bad, is called a good one.
Definition 4. A multi user code C = CX× CY is called a good multi user code, if both individual codebooks CX,
CY are good codes.
Definition 5. For a good multi user code C = CX×CY, and for a particular type PXY ∈ Pn(X × Y), we define
R(C,PXY) ?1
nlog|C ∩ TPXY|
(13)
Definition 6. For a sequence of joint types Pn
of type graphs, Gn, is defined as follows: For every n, Gnis a bipartite graph, with its left vertices consisting of
all xn∈ TPn
vertex on the right (say ˜ yn) if and only if (˜ xn, ˜ yn) ∈ TPn
XY∈ Pn(X × Y), with marginal types Pn
Xand Pn
Y, the sequence
Xand the right vertices consisting of all yn∈ TPn
Y. A vertex on the left (say ˜ xn) is connected to a
XY.
III. MAIN RESULT
The main result of this section is a new sphere packing bound for the average error probability for a
discrete memoryless multiple access channel. The idea behind the derivation of this bound is based on
the property that is common among all good multi user codes with certain rate pair. In the following, we
first derive a sphere packing bound for a good multiuser code. Next, we show that for any bad multiuser
code, there exists a good code with the same rate pair and smaller average probability of error. Therefore,
to obtain a lower bound for the average error probability for the best code, we only need to study good
codes (codes without any repeated codewords).
Now, consider a good multiuser code with blocklength n. Suppose the number of messages of the
first source is MX = 2nRXand the number of messages of the second source is MY = 2nRY. Assume
that all the messages of any source are equiprobable and the sources are sending data independently.
Considering these assumptions, all MXMY pairs are occuring with the equal probability. Thus, at the
October 8, 2010DRAFT

Page 5

5
input of the channel, we can see all possible 2n(RX+RY)(an exponentially increasing function of n) pairs
of input sequences. However, we also know that the number of possible types is a polynomial function
of n. Thus, for at least one joint type, the number of pairs of sequences in the multi user code sharing
that particular type, should be an exponential function of n with the rate arbitrary close to the rate of
the multi user code. We will look at these pairs of sequences as a subcode, and then try to find a lower
bound for the average error probability of this subcode. Following, we will show that this bound is a
valid lower bound for the average probability of error for the original code.
Lemma 1. [9] For any δ > 0, for any sufficiently large n, and for any good?n,2nRX,2NRY?multi user code C,
as defined above, there exists PXY ∈ Pn(X × Y) such that
R(C,PXY) ≥ RX+ RY − δ
for sufficiently large n,
PXY is called a dominant type of C.
Hence, for any good code, there must exist at least a joint type which dominates the codebook. We
can ask the following question: for a multiuser code, with rate (RX,RY), can any joint type potentially
be its dominant type? As shown later, the answer to this question helps us characterize a tighter sphere
packing bound. In response to this question, Nazari et al. [9] studied the type graphs for different joint
types and proved the following result:
Lemma 2. [9] For all sequences of nearly complete subgraphs of a particular type graph TPXY, the rates of the
subgraph (RX,RY) must satisfy
RX≤ H (X|U), RY ≤ H (Y |U)
(14)
for some PU|XY such that X − U − Y .
Now consider a particular joint type Pn
satisfying the constraint mentioned in lemma 2, the type graph corresponding to this joint type can not
contain an almost fully connected subgraph with rate (RX,RY). Consequently, it cannot be the dominant
type of a good multiuser code with rate (RX,RY).
XY. By the previous lemma, if there does not exist any PU|XY
Fact 1. Consider a good multiuser code C with parameter?n,2nRX,2nRY?. A joint type Pn
be the dominant type of C if there exists a PU|XY, X − U − Y , such that
XY∈ Pn(X × Y) can
RX≤ H (X|U), RY ≤ H (Y |U),
(15)
conversely, if it does not exist such a conditional distribution, then Pn
multiuser code with parameter?n,2nRX,2nRY?.
Theorem 1. Fix any RX≥ 0, RY ≥ 0, δ > 0 and a sufficiently large n. Consider a good multiuser code C with
parameter?n,2nRX,2nRY?which has a dominant type P∗
a code is bounded above by
XYcannot be the dominant type of any good
XY∈ Pn(X × Y). The average error exponent of such
Esp(RX,RY,W) ? min
VZ|XYD?VZ|XY||W|P∗
XY
?.
(16)
Here, the minimization is over all possible conditional distributions VZ|XY : X × Y → Z, which satisfy at least
October 8, 2010DRAFT