Minimum Terms of Residual Redundancy for Successful Iterative Source-Channel Decoding

Abstract

Iterative source-channel decoding (ISCD) improves the error robustness of a digital communication system by iteratively evaluating natural residual source redundancy and artificial channel coding redundancy in a TURBO-like process. Based on recent results to extrinsic information transfer (EXIT) charts we present a novel (experimental) approach to quantify the minimum terms of residual redundancy which are needed for (almost) successful ISCD. Moreover, we clarify why in certain situations the decoding trajectory exceeds the EXIT-characteristic of soft decision source decoding (SDSD) in an ISCD scheme

Full text

Page 1

778IEEE COMMUNICATIONS LETTERS, VOL. 10, NO. 11, NOVEMBER 2006
Minimum Terms of Residual Redundancy for
Successful Iterative Source-Channel Decoding
Marc Adrat, Thorsten Clevorn, Johannes Brauers, and Peter Vary
Abstract—Iterative source-channel decoding (ISCD) improves
the error robustness of a digital communication system by
iteratively evaluating natural residual source redundancy and
artificial channel coding redundancy in a TURBO-like process.
Based on recent results to extrinsic information transfer (EXIT)
charts we present a novel (experimental) approach to quantify
the minimum terms of residual redundancy which are needed
for (almost) successful ISCD. Moreover, we clarify why in certain
situations the decoding trajectory exceeds the EXIT-characteristic
of soft decision source decoding (SDSD) in an ISCD scheme.
Index Terms—Iterative Source-Channel Decoding (ISCD),
EXIT chart, Soft Decision Source Decoding (SDSD).
I. INTRODUCTION
M
signal. For instance, most state-of-the-art speech codecs are
based on the code excited linear prediction (CELP) principle
where the set of source codec parameters contains the spectral
coefficients, the indices and gain factors of the adaptive as well
as the fixed codebooks. Usually, due to complexity limitations
in the encoding process and due to design constraints, like the
maximum tolerable signal delay, these source codec parame-
ters exhibit considerable amounts of residual redundancy. Such
natural residual source redundancy might appear in terms of
a non-uniform parameter distribution or in terms of mutual
(time) dependencies among consecutive realizations.
Concepts how to benefit from residual redundancy in a
digital communication system in order to increase the error
robustness can be found (either stand-alone) for channel
decoding and for source decoding, or (jointly) for both.
The most famous approaches are known as source-controlled
channel decoding (SCCD) [1] and as softbit or soft decision
source-decoding (SDSD) [2]. The advantages of both concepts
can jointly be exploited by iterative source-channel decoding
(ISCD) [3]–[6]. In ISCD the natural residual source redun-
dancy and artificial channel coding redundancy are evaluated
iteratively in a TURBO-like process [7] (see Fig. 1). In order
to analyze the convergence behavior of TURBO-processes,
extrinsic information transfer (EXIT) charts have been intro-
duced [8].
In this letter, we take up recent results to EXIT-charts to
state a necessary, but not sufficient condition which can ensure
ODERN speech, audio, and video encoders extract a
set of source codec parameters from the original source
Manuscript received April 10, 2006. The associate editor coordinating the
review of this letter and approving it for publication was Dr. Stefano Buzzi.
This work was supported by the DFG (Deutsche Forschungsgemeinschaft).
M. Adrat is with the Research Establishment for Applied Sciences (FGAN
FKIE/KOM), Wachtberg, Germany (email: adrat@fgan.de).
T. Clevorn, J. Brauers, and P. Vary are with the Institute of Communication
Systems and Data Processing, RWTH Aachen University, Germany.
Digital Object Identifier 10.1109/LCOMM.2006.060544.
YSCCD
ASCCD
ESCCD
ASDSD
ESDSD
DSDSD
Π
Π−1
(De-)Interleaver
Source Contr.
Channel Dec.
Soft Decision
Source Dec.
Fig. 1.Exemplary iterative source-channel decoding (ISCD) scheme.
0 0.20.40.60.81
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
00.2 0.40.6 0.81
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
IE
SCCD, IA
SDSD
IE
SCCD, IA
SDSD
IA
SCCD, IE
SDSD
Not Successful ISCD
(Es/ N0 = − 3.8 dB)
Successful ISCD
(Es/ N0 = − 3.8 dB)
EXIT-Characteristic
of SDSD
EXIT-Characteristic
of SDSD
EXIT-Char.
of Chan. Dec.
EXIT-Characteristic
of Channel Decoding
Decoding Trajectory
Dec. Traject.
Fig. 2.Exemplary EXIT-charts and decoding trajectories for ISCD.
an (almost) error-free speech, audio, or video transmission if
using iterative source-channel decoding. This condition allows
to quantify the minimum terms of residual redundancy which
are required to apply ISCD successfully. Last but not least,
we analyze why in certain situations the decoding trajectory
exceeds the EXIT-characteristic of SDSD.
II. EXIT-CHART AND DECODING TRAJECTORY
In an EXIT-chart each component decoder of the TURBO-
process is represented by an EXIT-characteristic which depicts
the transfer of extrinsic information from the a priori input
A of the decoder to its extrinsic output E [8]–[10] (see
Fig. 1). For this purpose, the bitwise mutual information
IA= I(Vj;Aj) between the originally sent bits Vj, j =
1,...N (with N being the number of bits spent to encode a
source codec parameter), and the a priori information Aj, with
A = A1,...AN, is evaluated (resp. IE= I(Vj;Ej) in case
of the extrinsic output E). If the TURBO-process consists of
two components, the two respective EXIT-characteristics are
plotted into a common EXIT-chart regarding swapped axis
(because the extrinsic output of the one decoder serves as a
priori input for the other one, e.g. ASCCD↔ ESDSD, and vice
versa). Both EXIT-characteristics determine the bounds for
the so-called decoding trajectory [8]. The decoding trajectory
is a step-curve which visualizes the increase of extrinsic
information of both decoder components within the iterations.
Fig. 2 shows two exemplary EXIT-charts and decoding
trajectories if these are applied to the iterative source-channel
1089-7798/06$20.00 c ? 2006 IEEE

Page 2

ADRAT et al.: MINIMUM TERMS OF RESIDUAL REDUNDANCY FOR SUCCESSFUL ITERATIVE SOURCE-CHANNEL DECODING 779
decoding problem [5], [6], [10]–[12]. In both sub-plots the
EXIT-characteristic of the same terminated rate r = 1/2
recursive non-systematic convolutional (RNSC) code with
generator polynomial G(13/17,15/17)8is shown. Note, the
EXIT-characteristic of the channel code depends on the trans-
mission channel quality Es/N0 due to the channel-related
input YSCCD. Therein, Es denotes the energy per BPSK
modulated (binary phase shift keying) channel encoded bit and
N0/2 determines the power spectral density of the AWGN
(additive white Gaussian noise) transmission channel.
Moreover, each sub-plot shows an EXIT-characteristic of
a soft decision source decoder. In both cases, we considered
source codec parameters which are encoded using an Q = 16
level Lloyd-Max quantizer and an EXIT-optimized bit mapping
to N = 4 bit/level [6]. In the left case we assumed that
the source codec parameters exhibit residual redundancy in
terms of a Gaussian distribution (with zero mean and variance
σ2
auto-correlation is assumed to be ρ = 0.85. Obviously,
the intersection of the EXIT-characteristics, which marks a
limiting factor for the overall error correcting capability of
the TURBO-process, is at different (IE
The higher these values are, the higher the error correcting
capability can be. Thus, we will call the right sub-plot as (an
almost) successful approach to ISCD because there exists a
small tunnel through which the decoding trajectory can pass
to (relatively) high (IE
success, i.e., (IE
by ISCD because the EXIT-characteristic of SDSD is always
upper bounded to values IE
bounds for different settings of quantization concepts, bit
mapping strategies, and terms of auto-correlation have been
derived in [5], [6], [10], [11].
Two questions arise from the exemplary EXIT-charts and
decoding trajectories depicted in Fig. 2:
• Firstly, how much residual redundancy in terms of auto-
correlation is needed to allow (almost) successful ISCD?
• Secondly, why does in some situations the decoding
trajectory exceed the EXIT-characteristic of SDSD?
The answers will be given in the following.
V= 1) and auto-correlation ρ = 0.7. In the right case,
SCCD,IE
SDSD) values.
SCCD,IE
SDSD) = (1,1) can not be obtained
SDSD) values. However, absolute
SCCD,IE
SDSD,max< 1. The theoretical
III. MINIMUM TERMS OF RESIDUAL REDUNDANCY
Motivated by A. Ashikhmin’s analysis of universal proper-
ties of EXIT-charts [9] some additional, specific properties
have been derived for the EXIT-characteristic of the soft
decision source decoding component in an ISCD scheme
in [10]. One key observation of [10] was that the area under
the EXIT-characteristic of SDSD grows almost linearly with
the potential for data rate reduction ∆N. The data rate [10]
∆N = −1
can basically be saved if more efforts are spent to eliminate
all terms of auto-correlation ρ in the source codec parameter
flow, e.g, by linear prediction.
Figure 3 shows the EXIT-characteristics of SDSD for dif-
ferent values N ∈ {3,4,5} of bits/level and various values
ρ ∈ {0.0,0.7,0.8,0.85,0.9,0.95} of auto-correlation. In all
cases we used the EXIT-optimized bit mappings as proposed
2log2
?1 − ρ2?
(1)
0 0.2 0.40.6 0.81
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
SDSD
0.8
0.9
1
0 0.20.4 0.6
SDSD
0.81
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
00.20.40.6
SDSD
0.81
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
SDSD
0.8
0.9
1
00.511.5
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.45
0.5
N = 3 bit/level
EXIT-Optimized Bit Mapp.
N = 4 bit/level
EXIT-Optimized Bit Mapp.
N = 5 bit/level
EXIT-Optimized Bit Mapp.
Exemplary
ASDSD
ρ = 0 .0,... 0.95
ρ = 0 .0,... 0.95
ρ = 0 .0,... 0.95
Area ASDSD
∆ N
IE
SCCD,IA
IE
SCCD,IA
IA
SCCD,IE
IA
SCCD,IE
ρ = 0 .0
ρ = 0 .95
N = 3 ,4,5
0.659
0.781
0.938
Fig. 3. EXIT-characteristics of SDSD forρ ∈ {0.0,0.7,0.8,0.85,0.9,0.95}
and N = 3,4,5 bit/level; areas under these EXIT-characteristics.
in [6]. These mappings are designed such that they provide
a higher maximum value for IE
the EXIT-chart) than every other possible bit mapping. Note,
the EXIT-characteristics for N = 4 bit/level and ρ = 0.7 resp.
ρ = 0.85 correspond to the curves plotted in Fig. 2.
The sub-plot down right shows the linear dependencies on
∆N of the areas ASDSD under the EXIT-characteristics of
SDSD for the different settings of N and ρ.
A necessary, but not sufficient condition to ensure (al-
most) error-free iterative source-channel decoding is that
the area under the EXIT-characteristic of SDSD is greater
than the area under the swapped EXIT-characteristic of
SCCD, i.e., ASDSD> 1 − ASCCD(see Fig. 2).
In other words, if the area 1 − ASCCDis greater than ASDSD
then ISCD can never be successful. Thus, if a specific channel
coding scheme and a specific channel quality are given, e.g.,
the rate r = 1/2 RNSC code with G(13/17,15/17)8 at an
Es/N0= −3.8 dB (see Fig. 2) then we can determine the area
1 − ASCCD= 0.198. As a consequence, for successful ISCD
EXIT-characteristics of SDSD are required which exhibit an
area ASDSD > 0.198. The unsuitable region is marked gray
in the sub-plot down right in Fig. 3. The minimum residual
redundancy in terms of auto-correlation can be determined to
ρmin = 0.77 (resp. ∆N = 0.659) for N = 3 bit/level, to
ρmin= 0.81 (resp. ∆N = 0.781) for N = 4 bit/level, and to
ρmin= 0.85 (resp. ∆N = 0.938) for N = 5 bit/level.
Figure 4 shows the simulation results for the error correcting
capabilities. As quality criterion serves the parameter signal-
to-noise ratio (SNR) between the originally determined source
codec parameters and their reconstructed versions after ISCD.
For this purpose, the set of originally determined source codec
parameters is modeled by M = 500 independent 1st-order
Gauss-Markov processes with filter coefficient ρ. Thus, each
source codec parameter exhibits auto-correlation ρ, but there
is no cross-correlation between different parameters. Every set
of M codec parameters is individually channel encoded. The
SDSD,max(at the right border of

Page 3

780IEEE COMMUNICATIONS LETTERS, VOL. 10, NO. 11, NOVEMBER 2006
− 5
− 4
− 3
− 2
Es/ N0 [dB]
− 1
0
01
2
5
10
15
20
1stIteration
15thIteration
parameter SNR [dB]
10.29 dB
0.41 dB
Es/ N0 Under Test
(cmp . Fig. 2)
Fig. 4.Simulation results for ρ = 0.7 (dash-dotted) and ρ = 0.85 (solid).
remaining simulation settings for source and channel coding
correspond to the settings of Fig. 2. For details to the ISCD
algorithm we refer the reader to the literature, e.g. [3]–[6].
The dashed-dotted curves are the simulation results for ρ =
0.7 and the solid curves for ρ = 0.85. In both cases, significant
improvements can be obtained by ISCD (lower curve: 1st
iteration, upper curve: 15thiteration). However, at the channel
condition under consideration, i.e., at Es/N0= −3.8 dB, only
the solid curves reveal noteworthy SNR improvements by the
iterations. The reason is that ρ = 0.85 lies above the threshold
ρmin = 0.81. In contrast, an auto-correlation of ρ = 0.7
is by far not enough to make ISCD successful. Notice, the
corresponding EXIT-charts are shown in Fig. 2.
IV. DECODING TRAJECTORY EXCEEDS
EXIT-CHARACTERISTICS
Finally, it should be clarified why in certain situations the
decoding trajectory exceeds the EXIT-characteristic of the soft
decision source decoder (see right sub-plot of Fig. 2). The
reason can be found in the determination rule for the extrinsic
information of SDSD. The determination rule for a single bit
Ej
?
?
For convenience, we skipped the lower index “SDSD” of
Ej
the bit under test Vj
The outer summation in Eq. (2) has to be realized over all
2N−1possible permutations of V[j]
the inner summation runs over all 2Npermutations of Vτ−1at
time τ −1. The term θ(V[j]
?
The term P(V[j]
redundancy. It specifies the probability for a sent pattern Vτ
at time τ given the preceding pattern Vτ−1as well as a fixed
SDSD,τat time τ (corresponds to set index) reads [3], [5], [6]
?
θ(V[j]
Vτ−1
Ej
τ= log
V[j]
τ
θ(V[j]
τ)
Vτ−1
?
P(V[j]
τ|Vτ−1,Vj
τ= +1)α(Vτ−1)
V[j]
τ
τ)
P(V[j]
τ|Vτ−1,Vj
τ = −1)α(Vτ−1)
(2)
SDSD,τ. The upper index [j] in squared brackets means that
τis excluded from Vτ, i.e. V[j]
τ = Vτ\Vj
τ.
τ
of the present set τ and
τ) determines the input information
θ(V[j]
τ) = exp
i=1,...N,i?=j
Vi
2Ai
τ
SDSD,τ
.
(3)
τ|Vτ−1,Vj
τ) of Eq. (2) represents the residual
value for the bit under test Vj
preceding Vτ−1is taken into account by the rightmost term
α(Vτ−1). This term can be determined recursively over the
time instants in order to include the entire history [3], [5], [6].
The precision of the term α(Vτ−1) depends on the number
of iterations which were carried out in the preceding time
instants. Strictly speaking, this precision also depends on the
capabilities of the other constituent decoder in the TURBO-
scheme. To make the computation of the EXIT-characteristics
of SDSD independent from the channel coding component
it is common to assume, that only a single iteration will be
done. However, if larger numbers of iterations are performed
the precision of α(Vτ−1) is higher and thus, more extrinsic
mutual information IE
tory in Fig. 2 was depicted after the 15thiteration, it carries
more extrinsic mutual information and thus, it exceeds the
EXIT-characteristic of SDSD. We have verified by simulation
that the decoding trajectory meets the EXIT-characteristic of
SDSD if only a single iteration is carried out.
τ= ±1. The reliability for the
SDSDis available. As the decoding trajec-
V. CONCLUSIONS
Based on the area properties of EXIT-charts we presented
a novel (experimental) approach to quantify the minimum
terms of residual redundancy being required to make ISCD
successful. Moreover, we explained that due to the recursive
determination rule for the extrinsic information of SDSD its
EXIT-characteristic does not mark a tight upper bound for
the decoding trajectory. The latter aspect facilitates that even
(slightly) less residual redundancy can make ISCD successful.
REFERENCES
[1] J. Hagenauer, “Source-controlled channel decoding,” IEEE Trans. Com-
mun., vol. 43, no. 9, pp. 2449–2457, Sept. 1995.
[2] T. Fingscheidt and P. Vary, “Softbit speech decoding: a new approach to
error concealment,” IEEE Trans. Speech Audio Proc., vol. 9, pp. 240–
251, Mar. 2001.
[3] M. Adrat, P. Vary, and J. Spittka, “Iterative source-channel decoder
using extrinsic information from softbit-source decoding,” in Proc. IEEE
ICASSP 2001, vol. 4, pp. 2653–2656.
[4] N. G¨ ortz, “On the iterative approximation of optimal joint source-
channel decoding,” IEEE J. Sel. Areas Commun., vol. 19, pp. 1662–
1670, Sept. 2001.
[5] M. Adrat, “Iterative source-channel decoding for digital mobile com-
munications,” PhD thesis, Aachener Beitr¨ age zu Digitalen Nachrichten-
systemen (ed. P. Vary), ISBN 3-86073-835-6, RWTH Aachen, 2003.
[6] M. Adrat and P. Vary, “Iterative source-channel decoding: improved sys-
tem design using EXIT charts,” EURASIP J. Applied Signal Processing,
special issue on turbo processing, vol. 6, pp. 928-941, May 2005.
[7] C. Berrou and A. Glavieux, “Near optimum error correcting coding
and decoding: turbo-codes,” IEEE Trans. Commun., vol. 44, no. 10, pp.
1261-1271, Oct. 1996.
[8] S. ten Brink, “Convergence behavior of iteratively decoded parallel
concatenated codes,” IEEE Trans. Commun., vol. 40, pp. 1727–1737,
Oct. 2001.
[9] A. Ashikhmin, G. Kramer, and S. ten Brink, “Extrinsic information
transfer functions: model and erasure channel properties,” IEEE Trans.
Inf. Theory, vol. 50, pp. 2657–2673, Nov. 2004.
[10] M. Adrat, J. Brauers, T. Clevorn, and P. Vary, “The EXIT-characteristic
of softbit-source decoders,” IEEE Commun. Lett., vol. 9, pp. 540–542,
June 2005.
[11] M. Adrat, U. von Agris, and P. Vary, “Convergence behavior of iterative
source-channel decoding” in Proc. IEEE ICASSP 2003, vol. 4, pp. 269–
272.
[12] N. G¨ ortz and A. Schaefer, “The use of EXIT charts in iterative source-
channel decoding,” in Proc. EUSIPCO 2004, pp. 1549–1552.