Concatenated Coding for the AWGN Channel With Noisy Feedback

Abstract

The use of open-loop coding can be easily extended to a closed-loop concatenated code if the transmitter has access to feedback. This can be done by introducing a feedback transmission scheme as an inner code. In this paper, this process is investigated for the case when a linear feedback scheme is implemented as an inner code and, in particular, over an additive white Gaussian noise (AWGN) channel with noisy feedback. To begin, we look to derive an optimal linear feedback scheme by optimizing over the received signal-to-noise ratio (SNR). From this optimization, a linear feedback scheme is produced that is asymptotically optimal in the sense of blocklength-normalized SNR; it is then compared to other well-known schemes. Then, the linear feedback scheme is implemented as an inner code to a concatenated code over the AWGN channel with noisy feedback. This code shows improvements not only in error exponent bounds, but also in bit error rate (BER) and frame error rate (FER). It is also shown that if the concatenated code has total blocklength L and the inner code has blocklength, N , the inner code blocklength should scale as N = O (^C/_R), where C is the capacity of the channel and R is the rate of the concatenated code. Simulations with low-density parity-check (LDPC) and turbo codes are provided to display practical applications and their error rate benefits.

Full text

Page 1

arXiv:0909.0105v5 [cs.IT] 21 Feb 2011
1
Concatenated Coding for the AWGN Channel with
Noisy Feedback
Zachary Chance, Student Member, IEEE, and David J. Love, Senior Member, IEEE
Abstract—The use of open-loop coding can be easily extended
to a closed-loop concatenated code if the transmitter has access
to feedback. This can be done by introducing a feedback
transmission scheme as an inner code. In this paper, this process
is investigated for the case when a linear feedback scheme is
implemented as an inner code and, in particular, over an additive
white Gaussian noise (AWGN) channel with noisy feedback. To
begin, we look to derive an optimal linear feedback scheme
by optimizing over the received signal-to-noise ratio. From this
optimization, an asymptotically optimal linear feedback scheme
is produced and compared to other well-known schemes. Then,
the linear feedback scheme is implemented as an inner code
to a concatenated code over the AWGN channel with noisy
feedback. This code shows improvements not only in error
exponent bounds, but also in bit-error-rate and frame-error-
rate. It is also shown that if the concatenated code has total
blocklength L and the inner code has blocklength, N, the inner
code blocklength should scale as N = O?C
capacity of the channel and R is the rate of the concatenated
code. Simulations with low density parity check (LDPC) and
turbo codes are provided to display practical applications and
their error rate benefits.
R
?, where C is the
Index Terms—additive Gaussian noise channels, concatenated
coding, linear feedback, noisy feedback, Schalkwijk-Kailath cod-
ing scheme
I. INTRODUCTION
T
plementation of codes like turbo codes and low density parity
check (LDPC) codes. These codes have proven that open-loop
methods can be very powerful. However,an important question
to be asked is: “Can we do better with closed-loop coding?” In
this paper, we investigate the use of closed-loop concatenated
coding (see Fig. 1) over an additive white Gaussian noise
(AWGN) channel with noisy feedback. The benefits of this
have already been shown for a noiseless feedback channel as
in [1].
By definition, concatenated coding consists of two codes:
an inner code and an outer code. As for the outer code, we
will assume it is any general forward error-correction code as
to make this method applicable to any open-loop technique.
Furthermore, since we are interested in closed-loop coding,
the inner code will be a feedback transmission scheme; this,
HE field of open-loop error-correction coding has been
rich with innovations over the last 10-20 years with im-
This work was presented in part at the Asilomar Conference on Signal, Sys-
tems, and Computers in November 2009 and at the International Conference
on Acoustics, Speech, and Signal Processing in March 2010.
This work was supported in part by the National Science Foundation
under grant CCF0513916. Zachary Chance, a Ph.D. student, and David J.
Love, an associate professor, are affiliated with the School of Electrical
and Computer Engineering at Purdue University, West Lafayette, IN, 47907.
Email: zchance@purdue.edu, djlove@ecn.purdue.edu.
however, creates the intermediate goal of designing a good
feedback scheme. In this, we narrow our focus to the class
of linear feedback transmission schemes - meaning that each
transmission is a linear function of feedback side-information
and the message to be sent. With perfect feedback, this class
is known to have low complexity and high reliability [2], [3].
Therefore, we will try to exploit these advantages even with
a noisy feedback channel.
The search for the best linear feedback coding scheme for
AWGN channels has a long history, dating back to 1956 with
a paper by Elias [4]. However, most early work was done
in the 1960’s with papers like [5]–[12]. In 1966, Schalkwijk
and Kailath developed a specific linear coding technique that
utilizes a noiseless feedback channel [2], [3]. The coding
scheme was based off of a zero-finding algorithm called the
Robbins-Monro procedure which sequentially estimates the
zero of a function given noisy observations. Because of its low
complexity, much work has been done extending and evaluat-
ing the performance of the Schalkwijk-Kailath (S-K) scheme
in different circumstances. The performance was examined
when there is bounded noise on the feedback channel in [14].
In [15], [16], the system was observed under a peak energy
constraint. A generalization of the coding scheme for first-
order autoregressive noise processes on the forward channel
was derived in [5] while the problem was also looked at in
[17], [18]. The use of the coding technique was extended to
applications in stochastic controls in [6]. It was also extended
for use in stochastically degraded broadcast channels in [19]
and two-user Gaussian multiple access channels in [20]. The
scheme was used in [21] for a derivation of feedback capacity
for first-order moving average channels and, in general, for
channels with stationary Gaussian forward noise processes in
[22]. In [23], it was reformulated using a previous result in [4]
and then altered for specific use with PAM signaling. Varia-
tions on it were created by using stochastic approximation in
[24]. The S-K scheme was also used in a derivation of an
error exponent for AWGN channels with partial feedback in
[25]. This brief overview, of course, is not exhaustive as much
more literature can be found on the subject. In fact, due to the
notable popularity of the S-K scheme, we will implement it
for performance comparisons.
Recently, the area of general feedback communication
schemes has been also studied as in [26], [27]. These use a
technique called Posterior Matching in which information at
the receiver is refined using the a-posteriori density function
which is matched to the input distribution function. Such
techniques have also proven to be capacity-achieving and,
in fact, a generalization of the S-K scheme. However, these

Page 2

2
OUTER CODE
ENCODER
INNER CODE
ENCODER
INNER CODE
DECODER
OUTER CODE
DECODER
CONCATENATED
ENCODER
CONCATENATED
DECODER
NOISY FEEDBACK
CHANNEL
+
GAUSSIAN
NOISE
z[k]
x[k]
y[k]
Fig. 1.Closed-loop concatenated coding system.
schemes along with the S-K scheme all rely on the presence of
a noiseless feedback channel to achieve non-zero rate. If noise
is present, all of these schemes have only an achievable rate of
zero. However, coding with the presence of a noisy feedback
channel with variable length techniques has been investigated
in [28], [29].
In this paper, we do the following:
• Give the basic framework of concatenated coding and
design a linear feedback scheme for use as an inner code
by:
– Using a matrix formulation for feedback encoding,
we formulate the maximum SNR optimization prob-
lem. The formulation consists of a combining vector
and noise encoding matrix. It shares many similari-
ties to the method employed by [5]. In addition, an
upper bound on SNR for all noisy feedback schemes
over the AWGN channel is derived and shown to be
tighter than the bound previously made by [5].
– Using SNR as the cost function of interest, we solve
for (i) the SNR-maximizing linear receiver given
a fixed linear transmit encoding scheme and (ii)
the SNR-maximizing linear transmitter given a fixed
linear receiver.
– Using insights from the numerical optimization, we
derive what we believe to be the optimal linear
processing set-up. The performance of the proposed
scheme approaches the linear processing SNR upper
bound as the blocklength grows large.
• Using the proposed linear feedback scheme, we then
implement a concatenated code over the AWGN channel
with a general error-correction code as an outer code. The
error exponent for the concatenated scheme is derived in
terms of the error exponent for the outer code.
• Upper and lower bounds on the feedback error exponent
are then derived using this setup. These bounds are
then used to illustrate the effect of using the proposed
linear feedback scheme as an inner code. An approximate
trade-off between inner code blocklength and total code
blocklength is also derived.
• Simulations are run to show advantages in bit-error-rate
(BER) and frame-error-rate (FER) when the outer code
is either a turbo code or LDPC code.
The paper is organized in the following manner. The overall
system and the framework for a general closed-loop concate-
nated coding scheme are introduced in Section II. In Section
III, the concept of linear feedback coding is introduced to
develop an appropriate inner code for the overall concate-
nated scheme. Two methods of optimization for a general
linear coding scheme are also briefly introduced. Using these
optimization methods, a linear feedback scheme is proposed
in Section IV. This section also consists of analyzing the
asymptotic performance of our scheme, along with deriving
alternate proofs of results from related papers. In Section
V, the proposed linear feedback scheme is compared against
the S-K scheme to illustrate gains in performance. Section
VI introduces the concatenated coding scheme and its error
rate analyses. Simulations are then given in Section VII to
demonstrate practical concatenated code performance with
turbo codes and LDPC codes.
II. GENERAL CLOSED-LOOP CONCATENATED CODING
In this section, we formulate the general framework for a
closed-loop concatenated coding scheme; to begin, we look
specifically at the AWGN channel. At each channel use k =
1,2,...,L, the transmitted signal, x[k] ∈ R, is sent across the
channel. Likewise, the receiver obtains
y[k] = x[k] + z[k],
(1)
where, for our purposes, we will assume {z[k]} ∈ R are i.i.d.
such that z[k] ∼ N(0,1). Also, to remain practical, we can
impose an average transmit power constraint, ρ, such that
E[xTx] ≤ Lρ,
(2)
where x = [x[1],x[2],...,x[L]]T.
Consider sending a length K open-loop code across the
AWGN channel with noisy feedback. The transmission of
each component of the open-loop codeword, c ∈ RK, will be
encoded using an inner code of blocklength N that has access
to noisy feedback. Thus, the total concatenated codeword and
accordingly, the transmit vector, x, has length L = KN.
Note that the open-loop codeword is composed of entries
that lie on the real line; this implies that if the outer code
is binary, a modulation operation is implicit. Now, if we write
the components of the inner code as si(cj) (the i-th inner code
component used to encode the j-th outer code component),
then x can alternatively be written as
x = [s1(c1),s2(c1),...,sN(c1),s1(c2),...,sN(cK)].

Page 3

3
OUTER CODE
ENCODER
OUTER CODE
DECODER
+
GAUSSIAN
NOISE
z[i]
~
y[i]
ci
~
Fig. 2. Simplified concatenated coding scheme (˜ z[i] ∼ N(0,
ρ
SNRci)).
This encoding process now can be grouped by the concate-
nated encoder (or superencoder) in Fig. 1. At this point, we
also bound the average power of the outer codeword as
E[cTc] ≤ Kρ.
(3)
The codeword power constraint (3) will be useful when
we analyze the performance of the concatenated scheme in
Section VI. After all transmissions have been made, the inner
decoder creates an estimate of the current outer code codeword
by processing y, N entries at a time. This process produces
the following total codeword estimate
ˆ c = [ˆ c1,ˆ c2,...,ˆ cK],
(4)
which will be passed on to the outer decoder for final decod-
ing.
This setup now allows for a very convenient simplification
of the concatenated coding scheme. After processing at the
inner code decoder (which will be described in Section IV),
the channel given in (1) can be seen alternatively as
˜ y[i] = ci+ ˜ z[i],
(5)
as seen in Fig. 2 where the time index i is related to the original
channel use index as i = kN. The modified noise component,
˜ z[i], has a new variance dependent on the properties of the
inner code. In fact, the whole effect of the inner code is
encapsulated in the modified noise, ˜ z[i]. Due to the inner code
being undefined at this point, we cannot go into more depth.
However, a per-component signal-to-ratio can be calculated as
SNRci=
E[c2
E[(˜ z[i])2]=
i]
ρ
E[(˜ z[i])2].
(6)
Since the noise is i.i.d., this is the same for all ci. This implies
that V ar(˜ z[i]) =
This technique has converted the closed-loop problem now
into an open-loop problem. Note that this simplification is the
exact same process as defining the inner code and channel
together as a superchannel as discussed in [30]. Since the
outer code is a general forward error-correction code, this
equivalent mapping has greatly reduced the problem to now
finding the length N inner code that maximizes SNRciand,
thus, minimize the modified noise on the channel. In the next
few sections, this will be our exact focus.
ρ
SNRci.
III. THE INNER CODE: LINEAR FEEDBACK CODING
As stated above, the focus of this section is to design a
length N inner code that maximizes received SNR. Since
we have the availability of feedback for the inner code (see
Fig. 1) and it is a main focus of the paper, we will utilize
feedback side-information at the inner code encoder. With
this setup, it is possible to employ linear feedback encoding
- the advantages of which were described in the Section I.
To begin, the focus is now narrowed to only the inner code
encoder/decoder pair; hence, we will only be concerned with
sending and receiving one codeword of the inner code (i.e.,
[s1(ci),s2(ci),...,sN(ci)]). This corresponds to looking at
channel uses k = ((i − 1)N + 1),...,iN. For simplicity,
we study the case when i = 1. To begin, the notion of general
linear feedback coding is introduced. Because of the focus of
this section on the inner code and to ease with reference, we
to refer to the inner code encoder as the transmitter and the
inner code decoder as the receiver.
A. General Linear Feedback Encoding
In this section, we introduce the general framework of a lin-
ear feedback coding scheme in a linear algebraic formulation
(similar to [5]). A feedback channel allows the transmission
of data from the receiver back to the transmitter. Considering
the system in Fig. 3, we see that such a link is available with
unit delay and additive noise. As in Section II, at channel use
k = 1,2,...,N, x[k] is sent from the transmitter across an
AWGN channel and the receiver receives
y[k] = x[k] + z[k],
(7)
where {z[k]} are i.i.d. such that each z[k] ∼ N(0,1). Because
of the feedback channel, the transmitter also has access to
side-information. In this case, we assume the side-information
to be the past values of y[k] corrupted by additive noise,
n[k]. We assume that {n[k]} are i.i.d. such that n[k] ∼
N(0,σ2) and {n[k]} are independent of {z[k]} . Since we
are designing an encoding scheme that will utilize feedback,
x[k] is encoded at the transmitter using the noisy side in-
formation {y[1] + n[1],y[2] + n[2],...,y[k − 1] + n[k − 1]}.
By removing the known transmitted signal contribution,
this isequivalent toencoding with
{z[1] + n[1],z[2]+ n[2],...,z[k − 1] + n[k − 1]}.
We now describe a general coding scheme that utilizes this
channel and feedback configuration. The goal of the coding
scheme is to reliably send a component of the outer code
codeword, ci, from transmitter to receiver across an additive
noise channel using N channel uses. However, to broaden
the applications of the developed scheme, we look sending
a general message, θ ∈ R, instead of specifically ci. This
is possible due to the independent operation of the inner
code from the outer code. We assume the message symbol
θ is chosen from the set Θ = {θ1,θ2,...,θM} where M
is the number of symbols and each symbol is equally-likely.
Furthermore, we assume that θ is zero mean and that the
second moment of θ, E[θ2], is known. Due to the fact that
only received SNR and rate of transmission calculations will
be performed, the above description of the source alphabet
proves sufficient.
With this set-up, the input to the receiver can be written as
sideinformation
y = x + z,
(8)
where,
[x[1],x[2],...,x[N]]T. Because of the total average transmit
asabove,thenotation
x
refers to
x
=

Page 4

4
INNER CODE
ENCODER
(TRANSMITTER)
INNER CODE
DECODER
(RECEIVER)
+
OUTER CODE
COMPONENT
ci
x[k]
y[k]
FEEDBACK CHANNEL
y[k-1]
NOISE
z[k]
OUTER CODE
COMPONENT
ESTIMATE
ci
+
FEEDBACK NOISE
n[k-1]
Fig. 3.A transmitter/receiver pair over an AWGN channel with noisy feedback.
power constraint (2), the transmitted power of the signal x
(for N transmissions) is bounded by
E[xTx] ≤ Nρ.
(9)
The output of the transmitter x is given as
x = F(z + n) + gθ,
(10)
where g ∈ RNis a unit vector and F ∈ RN×Nis a matrix
called the encoding matrix. F is of the form
F =



0
···
...
...
···
···
0
...
...
0
f2,1
...
fN,1
...
fN,N−1



which is referred to as strictly lower-triangular to enforce
causality. Taking a closer look at (10), we see that this is
exactly the linear processing model - each x[k] is a linear
function of past values of {z[k] + n[k]} and the message, θ.
Now, consider the processing at the receiver’s end. The input
to the receiver y is given by (8). Using (10), (8) becomes
y = F(z + n) + gθ + z = (I + F)z + Fn + gθ.
(11)
After all N transmissions have been made, the receiver com-
bines all received values as a linear combination and forms an
estimate of the original message,ˆθ. This operation is written
as
ˆθ = qTy,
where q ∈ RNis a vector called the combining vector. It
is now evident that a general linear feedback scheme can be
completely described in terms of F,g, and q. In fact, the S-K
scheme can be described in this way, but since its definition
is not necessary, it will be pushed to Appendix A.
As an aside, it is important to note that up until this point,
a specific decoding process has not been specified. However,
since we will be passing on the output of the inner decoder
straight to the outer decoder, we choose only to perform only
soft decoding; hence the estimate will be sent straight to the
outer decoder without mapping it to an output alphabet. Of
course, minimum-distance decoding (and similar techniques)
can be easily implemented for hard decoding.
Looking back at the transmitted signal, it proves helpful to
study how much power is used sending the message and how
much is dedicated to encoding noise for noise-cancellation at
the receiver. This can be examined by noting that the average
transmitted power is
E[xTx]=
=
tr(FE[(z + n)(z + n)T]FT) + ?g?2E[θ2]
(1 + σ2)?F?2
?
?
Because the sum of the noise-cancellation power and signal
power must be less than Nρ, we introduce a new variable that
will be a measure of the amount of power used for noise-
cancellation. To accomplish this, let us introduce γ ∈ R such
that 0 ≤ γ ≤ 1. Using the power allocation factor γ, let E[θ2]
be scaled such that
F
???
noise-cancellation power
Nρ,
+ E[θ2]
? ?? ?
signal power
≤
where ?F?2
F=
i,j
f2
i,j.
E[θ2] = (1 − γ)Nρ,
(12)
and F be constrained such that
(1 + σ2)?F?2
F≤ Nγρ.
(13)
Until Section IV, it is now assumed that γ is fixed.
B. Optimization of Received SNR
As in Section II, our goal is to create a scheme that
maximizes the received signal-to-noise ratio. Not surprisingly,
we have chosen it to be our main performance metric. It can
be derived by noting the form of the receiver’s estimate of the
transmitted message. The received signal after combining is
ˆθ = qTy = qT((I + F)z + gθ + Fn).
(14)
It follows that the received SNR is
SNR =
E[??qTgθ??2]
E[θ2]??qTg??2
E[|qT(I + F)z + qTFn|2],
=
?qT(I + F)?2+ σ2?qTF?2.
(15)
It would be ideal to optimize the SNR expression over
all F,g, and q. However, this method turns out to be quite

Page 5

5
intractable. Instead, we focus on optimization by two con-
ditional optimization techniques that maximize SNR either
given F or given q. Since the derivations of these methods
are not necessary for our discussions, their proofs are pushed
to Appendix B. Note that the following procedures are hardly
groundbreaking, but are given as lemmas to aid in later refer-
ence. The first lemma is introduced to design F to maximize
the received SNR for a given q.
Lemma 1. Given a combining vector q and the power
constraint given in (13), the F that maximizes the received
SNR can be constructed using the following procedure:
1) Define q(i)= [qi+1,qi+2,...,qN]Twhere 1 ≤ i ≤ N −
1,
2) Construct the entries of F, fi,j, as
?
0,
fi,j=
−
qiqj
(1+σ2)?q(i)?
2+λ,i > j
i ≤ j
where λ ∈ R is the smallest λ ≥ 0 such that ?F?2
(1 + σ2)−1Nγρ.
F≤
The next lemma provides the symmetrical result; it constructs
a q that maximizes the received SNR given F.
Lemma 2. Given an encoding matrix, F, the q that maximizes
the received SNR can be found by letting q be the eigenvector
vector of (I + F)(I + F)T+ σ2FFTthat corresponds to its
minimum eigenvalue.
Note that this lemma has well-known analogous results in
estimation theory. In brief, the optimal q is created by forming
the projection
gTC−1
gTC−1g,
where C = (I + F)(I + F)T+ σ2FFT. However, it can be
shown that to choose g to maximize the received SNR given
F, then (16) implies that q = g and both should be chosen as
in Lemma 2. In addition, it is possible show that given F, the
definition of q in Lemma 2 produces the minimum variance
unbiased (MVU) estimator. To illustrate, the variance of the
estimator,ˆθ is
V ar(ˆθ) = E[(θ −ˆθ)2] = qT?(I + F)(I + F)T+ σ2FFT?q.
Since q has already been chosen to minimize this quantity
and it is unbiased (E[ˆθ] = θ), it is the MVU estimator
(consequentially also the least squares estimator).
These two lemmas will now prove sufficient for developing
a linear feedback scheme that maximizes the received SNR as
in Section IV.
qT=
(16)
C. Upper Bound on Rate and Received SNR
Due to the development of Lemmas 1 and 2, we can now
apply them to construct some interesting results on the class
of linear feedback codes. First, an upper bound on received
signal-to-noise ratio is found.
The method used in Lemma 1 to maximize the received
SNR focuses on minimizing the denominator of (15). It does
so while also compensating for the average power constraint
given in (9). If this constraint is relaxed to allow the denom-
inator of the SNR to be minimized completely, it is possible
to derive an upper bound on the received SNR.
Lemma 3. The received SNR for a linear feedback encoding
scheme with feedback noise variance, σ2, is bounded by
SNR ≤1 + σ2
σ2
Nρ
(17)
Proof: Looking at the proof of Lemma 1 (in Appendix B),
the goal is to maximize the received SNR by minimizing the
denominator in (15). However, the average power constraint
in (13) restricts the optimization problem and the solution is
not optimal in a least-squares sense. If the power constraint is
removed, (83) becomes
bopt= argmin
b
?Ab − q?2+ σ2?Ab?2.
(18)
This results in the solution to the least-squares problem
being
b = ((1 + σ2)ATA)−1ATq.
Using this b to construct F, (82) becomes
??qT(I + F)??2=
N−1
?
?
i=1
?
qi−
qi
1 + σ2
?2
+ q2
N
(19)
≥
σ2
1 + σ2
?2
.
(20)
Similarly, the other noise term is
??qTF??2=
N−1
?
≥
i=1
?
qi
1 + σ2
?2
+ q2
N
(21)
1
(1 + σ2)2.
(22)
Using these two results, the received SNR, using (15), can
be written as
SNR
≤
E?θ2?
+
?
1 + σ2
σ2
1 + σ2
σ2
σ2
1+σ2
?2
E?θ2?
Nρ
σ2
(1+σ2)2
(23)
=
(24)
≤
(25)
In [5], another upper bound is given for linear feedback
schemes with noise on the feedback channel. Using the
notation consistent with the above formulations, the Butman
bound can be given by:
SNR
≤
E?xTx?+
1 + σ2
σ2
1
σ2E?yTy?,
Nρ + 2?F?2
(26)
=
F+N
σ2.
(27)
Since the last two terms in the inequality are strictly greater
than zero, this bound is strictly greater than (17), implying a
helpful tightness in the new bound in Lemma 3.
Suppose that we now allow the size of the symbol set,
Θ = {θ1,θ2,...,θM}, to be a function of the blocklength