Page 1

arXiv:0708.0271v1 [cs.IT] 2 Aug 2007

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, AUG. 2007.1

Capacity Region of the Finite-State Multiple

Access Channel with and without Feedback

Haim Permuter and Tsachy Weissman

Abstract

The capacity region of the Finite-State Multiple Access Channel (FS-MAC) with feedback that may be an

arbitrary time-invariant function of the channel output samples is considered. We characterize both an inner and an

outer bound for this region, using Masseys’s directed information. These bounds are shown to coincide, and hence

yield the capacity region, of FS-MACs where the state process is stationary and ergodic and not affected by the

inputs. Though ‘multi-letter’ in general, our results yield explicit conclusions when applied to specific scenarios of

interest. E.g., our results allow us to:

• Identify a large class of FS-MACs, that includes the additive mod-2 noise MAC where the noise may have

memory, for which feedback does not enlarge the capacity region.

• Deduce that, for a general FS-MAC with states that are not affected by the input, if the capacity (region) without

feedback is zero, then so is the capacity (region) with feedback.

• Deduce that the capacity region of a MAC that can be decomposed into a ‘multiplexer’ concatenated by a point-

to-point channel (with, without, or with partial feedback), the capacity region is given byP

mRm ≤ C, where

C is the capacity of the point to point channel and m indexes the encoders. Moreover, we show that for this

family of channels source-channel coding separation holds.

Index Terms

Feedback capacity, multiple access channel, capacity region, directed information, causal conditioning, code-tree,

source-channel coding separation, sup-additivity of sets.

I. INTRODUCTION

The Multiple Access Channel (MAC) has received much attention in the literature. To put our contributions

in context, we begin by briefly describing some of the key results in the area. The capacity region for the

memoryless MAC was derived by Ahlswede in [1]. Cover and Leung derived an achievable region for a memoryless

MAC with feedback in [2]. Using block Markov encoding, superposition and list codes, they showed that the

region R1 ≤ I(X1;Y |X2,U), R2 ≤ I(X2;Y |X1,U) and R1+ R2 ≤ I(X1,X2;Y ) where P(u,x1,x2,y) =

p(u)p(x1|u)p(x2|u)p(y|x1,x2) is achievable for a memoryless MAC with feedback. Willems showed in [3] that

The authors are with the Department of Electrical Engineering, Stanford University, Stanford, CA 94305, USA. (Email: {haim1,

tsachy}@stanford.edu)

This work was supported by the NSF through the CAREER award and TFR-0729119 grant.

Page 2

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, AUG. 2007.2

the achievable region given by Cover and Leung for a memoryless channel with feedback is optimal for a class of

channels where one of the inputs is a deterministic function of the output and the other input. More recently Bross

and Lapidoth [4] improved Cover and Leung’s region, and Wu et. al. [5] have extended Cover and Leung’s region

for the case that non-causal state information is available at both encoders.

Ozarow derived the capacity of a memoryless Gaussian MAC with feedback in [6], and showed it to be achievable

via a modification of the Schalkwijk-Kailath scheme [7]. In general, the capacity in the presence of noisy feedback

is an open question for the point-to-point channel and a fortiori for the MAC. Lapidoth and Wigger [8] presented an

achievable region for the case of the Gaussian MAC with noisy feedback and showed that it converges to Ozarow’s

noiseless-feedback sum-rate capacity as the feedback-noise variance tends to zero. Other recent variations on the

Schalkwijk-Kailath scheme of relevance to the themes of our work include the case of quantization noise in the

feedback link [9] and the case of interference known non-causally at the transmitter [10].

Verd´ u characterized the capacity region of a Multi-Access channel of the form P(yi|xi

P(yi|xi

synchronism between the two users, i.e., there is a random shift between the users, only stationary input distributions

1,xi

2,yi−1)=

1,i−m,xi

2,i−m) without feedback in [11]. Verd´ u further showed in that work that in the absence of frame

need be considered. Cheng and Verd´ u built on the capacity result from [11] in [12] to show that for a Gaussian

MAC there exists a water-filling solution that generalizes the point-to-point Gaussian channel.

In [13] [14], Kramer derived several capacity results for discrete memoryless networks with feedback. By using

the idea of code-trees instead of code-words, Kramer derived a ‘mulit-letter’ expression for the capacity of the

discrete memoryless MAC. One of the main results we develop in the present paper extends Kramer’s capacity

result to the case of a stationary and ergodic Markov Finite-State MAC (FS-MAC), to be formally defined below.

In [15] [16], Han used the information-spectrum method in order to derive the capacity of a general MAC

without feedback, when the channel transition probabilities are arbitrary for every n symbols. Han also considered

the additive mod-q MAC, which we shall use here to illustrate the way in which our general results characterize

special cases of interest. In particular, our results will imply that feedback does not increase the capacity region of

the additive mod-q MAC.

In this work, we consider the capacity region of the Finite-State Multiple Access Channel (FS-MAC), with

feedback that may be an arbitrary time-invariant function of the channel output samples. We characterize both an

inner and an outer bound for this region. We further show that these bounds coincide, and hence yield the capacity

region, for the important subfamily of FS-MACs with states that evolve independently of the channel inputs. Our

derivation of the capacity region is rooted in the derivation of the capacity of finite-state channels in Gallager’s

book [17, ch 4,5]. More recently, Lapidoth and Telatar [18] have used it in order to derive the capacity of a

compound channel without feedback, where the compound channel consists of a family of finite-state channels. In

particular, they have introduced into Gallager’s proof the idea of concatenating codewords, which we extend here

to concatenating code-trees.

Though ‘multi-letter’ in general, our results yield explicit conclusions when applied to more specific families

of MACs. For example, we find that feedback does not increase the capacity of the mod-q additive noise MAC

Page 3

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, AUG. 2007.3

(where q is the size of the common alphabet of the input, output and noise), regardless of the memory in the

noise. This result is in sharp contrast with the finding of Gaarder and Wolf in [19] that feedback can increase the

capacity even of a memoryless MAC due to cooperation between senders that it can create. Our result should also

be considered in light of Alajaji’s work [20], where it was shown that feedback does not increase the capacity of

discrete point-to-point channels with mod-q additive noise. Thus, this part of our contribution can be considered

a multi-terminal extension of Alajaji’s result. Our results will in fact allow us to identify a class of MACs larger

than that of the mod-q additive noise MAC for which feedback does not enlarge the capacity region.

Further specialization of the results will allow us to deduce that, for a general FS-MAC with states that are

not affected by the input, if the capacity (region) without feedback is zero, then so is the capacity (region) with

feedback. It will also allow us to identify a large class of FS-MACs for which source-channel coding separation

holds.

The remainder of this paper is organized as follows. We concretely describe our channel model and assumptions

in Section II. In Section III we introduce some notation, tools and results pertaining to directed information and the

notion of causal conditioning that will be key in later sections. We state our main results in Section IV. In Section V

we apply the general results of Section IV to obtain the capacity region for several interesting classes of channels,

as well as establish a source-channel separation result. The validity of our inner and outer bounds is established,

respectively, in Section VI and Section VII. In Section VIII we show that our inner and outer bounds coincide,

and hence yield the capacity region, when applied to the FS-MAC without feedback. This result can be thought

of as the natural extension of Gallager’s results [17, Ch. 4] to the MAC or, alternatively, as the natural extension

of Gallager’s derivation of the MAC capacity region in [21] to channels with states. In Section IX we characterize

the capacity region for the case of arbitrary (time-invariant) feedback and FS-MAC channels with states that evolve

independently of the input, as well as the FS-MAC with limited ISI (which is the natural MAC-analogue of Kim’s

point-to-point channel [22]), by showing that our inner and outer bounds coincide for this case. We conclude in

Section X with a summary of our contribution and a related future research direction.

II. CHANNEL MODEL

In this paper, we consider an FS-MAC (Finite state MAC) with a time invariant feedback as illustrated in Fig. 1.

The MAC setting consists of two senders and one receiver. Each sender l ∈ {1,2} chooses an index mluniformly

from the set {1,...,2nRl} and independently of the other sender. The input to the channel from encoder l is

denoted by {Xl1,Xl2,Xl3,...}, and the output of the channel is denoted by {Y1,Y2,Y3,...}. The state at time i,

i.e., Si ∈ S, takes values in a finite set of possible states. The channel is stationary and is characterized by a

conditional probability P(yi,si|x1i,x2i,si−1) that satisfies

P(yi,si|xi

1,xi

2,si−1,yi−1) = P(yi,si|x1i,x2i,si−1),

(1)

where the superscripts denote sequences in the following way: xi

l= (xl1,xl2,...,xli), l ∈ {1,2}. We assume a

communication with feedback zi

lwhere the element zliis a time-invariant function of the output yi. For example,

Page 4

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, AUG. 2007.4

Encoder 1

x1,i(m1,zi−1

1

)

Encoder 2

x2,i(m2,zi−1

2

)

m1

∈ {1,...,2nR1}

m2

∈ {1,...,2nR2}

Finite State MAC

P(yi,si|x1,i,x2,i,si−1)

Time-Invariant

Function

Time-Invariant

Function

z2,i(yi)

z1,i(yi)

z2,i−1

z1,i−1

Decoder

Unit

Delay

Unit

Delay

ˆ m1(yN)

ˆ m2(yN)

ˆ m1, ˆ m2

yi

yi

Fig. 1. Channel with feedback that is a time invariant deterministic function of the output.

zlicould equal yi(perfect feedback), or a quantized version of yi, or null (no feedback). The encoders receive the

feedback samples with one unit delay.

A code with feedback consists of two encoding functions gl: {1,...,2nR1} × Zn−1

kth coordinate of xn

l

→ Xn

l, l = 1,2, where the

l∈ Xn

lis given by the function

xlk= glk(ml,zk−1

l

),k = 1,2,...,n,l = 1,2

(2)

and a decoding function,

g : Yn→ {1,...,2nR1} × {1,...,2nR2}.

(3)

The average probability of error for ((2nR1,2nR2,n) code is defined as

P(n)

e

=

1

2n(R1+R2)

?

w1,w2

Pr{g(Yn) ?= (w1,w2)|(w1,w2) sent}.

(4)

A rate (R1,R2) is said to be achievable for the MAC if there exists a sequence of ((2nR1,2nR2),n) codes with

P(n)

e

→ 0. The capacity region of MAC is the closure of the set of achievebale (R1,R2) rates.

III. DIRECTED INFORMATION

Throughout this paper we use the Causal Conditioning notation (·||·). We denote the probability mass function

(pmf) of YNcausally conditioned on XN−d, for some integer d ≥ 0, as P(yN||xN−d) which is defined as

N

?

(if i − d ≤ 0 then xi−dis set to null). In particular, we extensively use the cases where d = 0,1:

N

?

P(yN||xN−d) ?

i=1

P(yi|yi−1,xi−d),

(5)

P(yN||xN) ?

i=1

P(yi|yi−1,xi)

(6)

Page 5

SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, AUG. 2007.5

Q(xN||yN−1) ?

N

?

i=1

Q(xi|xi−1,yi−1),

(7)

where the letters Q and P are both used for denoting pmfs.

Directed information I(XN→ YN) was defined by Massey in [23] as

I(XN→ YN) ?

N

?

i=1

I(Xi;Yi|Yi−1).

(8)

It has been widely used in the characterization of capacity of point-to-point channels [22], [24]–[29], compound

channels [30], network capacity [14], [31], rate distortion [32]–[34] and computational biology [35], [36]. Directed

information can also be expressed in terms of causal conditioning as

I(XN→ YN) =

N

?

i=1

I(Xi;Yi|Yi−1) = E

?

logP(YN||XN)

P(YN)

?

,

(9)

where E denotes expectation. The directed information from XNto YN, conditioned on S, is denoted as I(XN→

YN|S) and is defined as:

I(XN→ YN|S) ?

i=1

N

?

I(Xi;Yi|Yi−1,S).

(10)

Directed information between XN

1 to YNcausally conditioned on XN

2 is defined as

I(XN

1→ YN||XN

2) ?

N

?

i=1

I(Xi

1;Yi|Xi

2,Yi−1) = E

?

logP(YN||XN

P(YN||XN

1,XN

2)

2)

?

.

(11)

where P(yN||xN

Throughout this paper we are using several properties of causal conditioning and directed information that follow

1,xN

2) =?N

i=1P(yi|yi−1,xi

1,xi

2).

from the definitions and simple algebra. Many of the key properties that hold for mutual information and regular

conditioning carry over to directed information and causal conditioning, where P(xN) is replaced by P(xN||yN−1)

and P(yN) is replaced by P(yN||xN). Specifically,

Lemma 1: (Analogue to P(xN

1,yN) = P(xN

1)P(yN|xN

1).) For arbitrary random vectors (XN

1,XN

2,YN),

P(xN

1,yN) = P(xN

1||yN−1)P(yN||xN

1)

(12)

P(xN

1,yN||xN

1;YN) − I(XN

2) = P(xN

1||yN−1,xN

1;YN|S)| ≤ H(S).) For arbitrary random vectors and variables,

2)P(yN||xN

1,xN

2).

(13)

Lemma 2: (Analogue to |I(XN

??I(XN

1→ YN||XN

1→ YN) − I(XN

1→ YN|S)??≤ H(S) ≤ log|S|

2) − I(XN

(14)

??I(XN

1→ YN||XN

2,S)??≤ H(S) ≤ log|S|.

(15)

The proofs of Lemma 1 and Lemma 2 can be found in [27, Sec. IV], along with some additional properties of causal

conditioning and directed information. The next lemma, which is proven in Appendix I, shows that by replacing

regular pmf with causal conditioning pmf we get the directed information. Let us denote the mutual informa-

tion I(Xn

1;Yn|Xn

2) as a functional of Q(xN

1,xN

2) and P(yN|xN

1,xN

2), i.e., I(Q(xN

1,xN

2);P(yN|xN

1,xN

2)) ?