A Note on Comparison of Error Correction Codes

Abstract

Use of an error correction code in a given transmission channel can be regarded as the statistical experiment. Therefore, powerful results from the theory of comparison of experiments can be applied to compare the performances of different error correction codes. We present results on the comparison of block error correction codes using the representation of error correction code as a linear experiment. In this case the code comparison is based on the Loewner matrix ordering of respective code matrices. Next, we demonstrate the bit-error rate code performance comparison based on the representation of the codes as dichotomies, in which case the comparison is based on the matrix majorization ordering of their respective equivalent code matrices.

Full text

Page 1

arXiv:0710.3817v1 [cs.IT] 20 Oct 2007
A Note on Comparison of Error Correction Codes
Dejan V. Djonin
Abstract
Use of an error correction code in a given transmission channel can be regarded as the statistical experi-
ment. Therefore, powerful results from the theory of comparison of experiments can be applied to compare the
performances of different error correction codes. We present results on the comparison of block error correction
codes using the representation of error correction code as a linear experiment. In this case the code comparison is
based on the Loewner matrix ordering of respective code matrices. Next, we demonstrate the bit-error rate code
performance comparison based on the representation of the codes as dichotomies, in which case the comparison
is based on the matrix majorization ordering of their respective equivalent code matrices.
Index Terms
Deficiency distance, error-correction code, code design, probability of error, Bayes risk, zonotope, matrix
majorization.
I. INTRODUCTION
In this section we will review the basic concepts of statistical experiments that will later be used to
establish ordering relation between error correction codes employed in a given communication channel.
Following [1], a statistical experiment is defined as a pair E = (X,Pθ;θ ∈ Θ) where X is a measurable
sample space, Pθis a probability measure on X for each θ ∈ Θ and Θ is a parameter set. Let the decision
rule y = σ(x) of experiment E be defined as a mapping from an element of the sample space x ∈ X to
the element of the decision space y ∈ Y. Further, one can define the loss function L(θ,y) of choosing
the decision y ∈ Y when the true state of the parameter is θ ∈ Θ. The reader is referred to [1] for
a thorough treatment of statistical experiments, loss functions, with numerous illustrative examples. We
will next introduce the notation used throughout the paper, followed by the definition and properties of
deficiency distance between experiments.
The author is with Dyaptive Systems, Inc., Vancouver, BC and adjunct professor at the Department of Electrical and Computer Engineering,
University of British Columbia, 2356 Main Mall, Vancouver, BC, V6T 1Z4, Canada, e-mail: ddjonin@ece.ubc.ca.

Page 2

Notation: We will use a′to denote the transpose of vector a, A+denotes the Moore-Penrose pseudoin-
verse of the matrix A, #S denotes the cardinality of set S. A ∧ B and A ∨ B denote the element-wise
minimum and maximum of two equal dimensional matrices A and B, respectively. Let A(i,:) and A(:,i)
denote the i-th row and i-th column of the matrix A, respectively. Range range(B) of the matrix B is the
space spanned by the columns of the matrix B. Let eibe [0,0,...,0,1,0,...,0] where 1 is at the i-th
position of vector ei. Let conv(a,b) = {αa + (1 − α)b|α ∈ [0,1]} for equal dimensional vectors a and
b. Let Mnmbe the set of row-stochastic n × m dimensional matrices, i.e. these matrices have positive
elements whose rows sum to 1. Let indicator function I{x}be equal to 1 if x is true and 0 otherwise.
N(µ,Σ) denotes the multivariate normal random variable with mean µ and covariance Σ. ?f? denotes the
total variation of function f(x) given by?|f(x)|dx. Euis the expectation with respect to (w.r.t) random
variable u.
Le Cam has introduced in [2] the following definition of the deficiency distance between two experi-
ments:
Definition 1: (Deficiency Distance) Experiment E = (X,Pθ;θ ∈ Θ) is ǫ-deficient w.r.t experiment
F = (Y,Qθ;θ ∈ Θ) if for each Θ0 ⊂ Θ, and each decision rule ρ of experiment F, there exists a
decision rule σ of experiment E such that
?
Pθ(x)L(θ,σ(x))dx ≤
?
Qθ(y)L(θ,ρ(y))dy + ǫθmax
y
L(θ,y)
(1)
when θ ∈ Θ0. In short form, the deficiency distance is denoted as ǫ = δ(E,F).
The deficiency distance can plausibly be interpreted as follows: the deficiency of experiment E w.r.t
the experiment F is the upper bound on the difference in the risk functions1between experiments E and
F for any choice of the a priori knowledge of the unknown parameter θ and some decision rule on E. In
general, therefore, the deficiency distance is not symmetric, i.e. δ(E,F) ?= δ(F,E).
Historically, the deficiency distance is a generalization of the older concept of sufficiency ordering of
experiments.
Definition 2: (Sufficient Experiments [3], [1]) Experiment E = (X;Pθ,θ ∈ Θ) is sufficient for exper-
iment F = (Y;Qθ,θ ∈ Θ) iff δ(E,F) = 0. The ordering of experiments based on sufficiency is also
denoted by E ≥SF.
1Expression
RPθ(x)L(θ,σ(x))dx is commonly referred as the risk function of the experiment E = (X,Pθ;θ ∈ Θ) for a given loss
function L and decision rule σ.
2

Page 3

Note that ordering ≥Sis a partial ordering on the set of experiments with the same parameter set. This
ordering is sometimes also referred to as Blackwell ordering. We next discuss the basic properties of the
deficiency distance.
Theorem 1: (Properties of Deficiency Distance) Let E,F,G be experiments. Then
(i) 0 ≤ δ(E,F) ≤ 2 − 2(#Θ)−1
(ii) δ(E,G) ≤ δ(E,F) + δ(F,G)
iii) δ(E,E) = 0
Proof: Theorem 6.2.24 in [1]. ?
A special case of a statistical experiment is a linear experiment defined below2:
Definition 3: (Linear Normal Experiment) Linear normal experiment E is denoted as E(A,Σ;β ∈ Rn)
and is experiment with the sample distribution N(A′β,Σ) where the parameter is β ∈ Rnand the
parameter set is Rn. Real valued matrices A and Σ are assumed to be known.
Two linear experiments can be compared as follows.
Theorem 2: (Linear Experiment Comparison) (a) Linear normal experiment E(A,Σ;β ∈ Rn) is suffi-
cient for F(B,Γ;β ∈ Rn) (i.e. δ(E,F) = 0) if
AΣA′− BΓB′is non-negative definite matrix.(2)
(b) Let Σ = I and Γ = I then δ(E,F) < 2 iff range(B) ⊆ range(A). Furthermore, if range(B) ⊆
range(A) then
δ(E,F) = ?N(0,(B′(AA′)+B) ∨ I) − N(0,I)?.
(3)
Proof: Theorem 8.2.13 and 8.5.7 in [1]. ?
General statistical results and inequalities can provide important insights into the operation and efficient
design of a communication system. An early result on the influence of the communication channels based
on Blackwell ordering of channel transfer matrices has been presented [4]. More recently, monotonicity
results on the influence of Rician fading on the capacity of MIMO systems has been demonstrated in [5].
Based on general inequalites of stochastic majorization, performance comparison of various receivers in
multipath fading channels and the influence of power delay profile has been shown in [6].
Starting with the general notion of deficiency, we will present in the next section the application of
these concepts to the comparison of error correction codes. Namely, in Subsection II-A we present the
2A more general definition of linear experiment may be given as in [1], Section 8.
3

Page 4

block error correction code comparison in additive white Gaussian Noise (AWGN) channels based on
linear experiments and block-error rates. In Subsection II-B we present the bit-error rate comparison of
error-correction codes used in discrete channels based on matrix majorization.
II. ERROR CORRECTION BLOCK CODE COMPARISON
Let (M,n) block error correction code denoted with C be defined as a map φC: i → xifrom the set
i ∈ Θ = {1,...,M} of possible information messages to the set of codewords xi∈ Anfor i = 1,...,M.
Let A denote the alphabet of the code symbols.
Let the received message be z if the transmitted code word is x = φC(y) as a result of encoding the
information message y. The dependence of received message z on x is described with a probabilistic law,
examples of which will be discussed in more detail in the following sections.
The original transmitted message y is recovered through the usage of decoder δ which maps the received
message z to a possible transmitted data message y
δ(z) → y.
(4)
In the information theory as well as communications practice, a common metric used to evaluate the
performance of the block error correction code is the packet (code-word) error probability. Let the
transmitted message be y. Then, the packet error probability of code C is equal to
PC
e(y) = E?I{δ(z)?=y}
?
(5)
where expectation is over realizations of the random variable z of the received code word given transmitted
data message y.
A. Comparison of Error Correction Codes in AWGN Channels Based on Linear Normal Experiments
In this section we will address the transmission of the coded message over an additive white Gaussian
noise channel. Namely, assume that code word x ∈ Rnis being transmitted and that the alphabet of
the code symbols is R. The code words are assumed to be energy bounded such that xx′≤ E where
parameter E is the upper bound on the energy of the code word. The received data is given as
z = x + ν
(6)
4

Page 5

TABLE I
EQUIVALENT TERMINOLOGY USED IN STATISTICAL EXPERIMENTS AND ERROR-CORRECTION CODES
Statistical ExperimentsError-Correction Codes
parameter θ
information message
decision ruledecoding algorithm
sample spaceset of received code words
loss functionrate-distortion measure
where ν is the additive Gaussian noise with covariance matrix E[νν′] = σ2I. The signal to noise ratio is
therefore upper bounded with
E
nσ2.
For transmission in additive white Gaussian noise channel, a block error correction code can be
interpreted as a linear experiment E(A,Iσ2;β ∈ RM) such that the parameter set is Θ = RMand matrix
is A = [x1x2...xM]′. The i-th information message is represented with the parameter β = ei,i = 1,...M.
Matrix A that uniquely describes such a block code will be called the code matrix and the adjoint block
code will be denoted in short with A. In the context of the comparison of experiments, decoder δ can be
interpreted as the decision rule, while I{δ(z)?=β}is the loss function, for a received message z and transmitted
data message β. A comparison of error-correction code and statistical experiments terminology is shown
in Table I for easy reference.
Proposition 1: (Comparison of two codes based on the packet error probability) Let A and B be two
code matrices that define block error correction codes A and B respectively. (a) If noise variance σ2is a
known parameter and if
AA′− BB′is non-negative definite matrix (7)
then there exists a decoder of the code A that will always have smaller bit error rate than any decoder of
the code B
PA
e(β) ≤ PB
e(β)
(8)
for any transmitted data message β = ei,i ∈ 1,...,M.
(b) If the above condition is not satisfied and if range(B) ⊆ range(A) then for any data message β
PA
e(β) − PB
e(β) ≤ δ(A,B) = ?N(0,(B′(AA′)+B) ∨ I) − N(0,I)?
(9)
where PA
e(β) and PB
e(β) are packet error rates of block codes A and B respectively.
5