Page 1
CUTOFF RATE AND SIGNAL DESIGN FOR THE RAYLEIGH FADING SPACETIME
CHANNEL
Alfred 0. Hero *
Dept. EECS
University of Michigan
130 1 Beal Avenue
Ann Arbor, MI 48 1092 122
hero@eecs.umich.edu
ABSTRACT
We consider the singleuser computational cutoff rate for
the complex Rayleigh flat fading spatiotemporal channel
under a peak power constraint. Determination of the cut
off rate requires maximization of an average error expo
nent over all possible spacetime codeword probability dis
tributions. This error exponent is monotone decreasing in
a measure of dissimilarity between pairs of codeword ma
trices. For low SNR the dissimilarity function reduces to a
trace norm of differences between outerproducts of pairs
of codewords. We characterize the cutoffrate and the rate
achieving constellation under different operating regimes
depending on the number o f transmit and receive anten
nas, the number of codewords in the constellation, and the
received SNR.
1. INTRODUCTION
In this paper we investigate the cutoff rate for the Rayleigh
flat fading spatiotemporal channel model introduced by Marzetta
and Hochwald [5] under a maximum peak transmitted power
constraint. Codewords {Si} for this channel are complex
T x M matrices whose T rows represent temporal coordi
nates and whose M columns represent spatial coordinates,
indexed over T transmitted time samples and M transmitter
antenna, respectively. The set of peak constrained code
words s are the set Of
satisfy the peak power constraint: llsi112 5 TM, where
Thomas L. Marzetta
Mathematical Sciences Research Center
Bell Laboratories
Lucent Technologies
600 Mountain Avenue
Murray Hill, NJ 07974
tlm@research.belllabs.com
ifies the highest information rate beyond which sequential
decoding becomes impractical [6, 81 and as it is frequently
simpler to calculate than channel capacity.
The receiver is an N element antenna array which, for L
transmitted codeword matrices
the sequence of T x N observation matrices
Xi = JsjSiHi + Wi, i = 1,. . . , L
Si E S, produces
(1)
where r) = p/M is the normalized signaltonoise ratio
(SNR) with p > 0 the SNR perentry of Si, Hi is an M x N
matrix of complex channel coefficients, and Wi is a T x N
matrix of complex noises. The piecewise constant Rayleigh
flat fading model corresponds to taking the LN(T + M)
elements of the matrices
zero mean Gaussian random variables with unit variance.
The following results are presented. Proofs are given in [3].
and {Wi}t.l to be i.i.d.
For the flat Rayleigh fading model under a peak trans
mitted power constraint there is no advantage to using more
transmitting antennas than time samples (Proposition 1). Fur
thermore, there is no advantage to transmitting signals that
not spatially orthogonal, i.e. one might as well transmit
mutually oflhogonal temporal waveforms at each antenna
element. These Parallel results of Marzetta and Hochwald
for average Power constraints [51*
matrices which
llS112 = tr{SSH}.
An integral representation for the cutoff rate is obtained
which depends on a pairwise dissimilarity
set of signal matrices. n i S dissimilarity
creasing function of the spatial correlation between pairs of
signa] matrices. For low SNR the dissimilarity measure re
Over the
is a de
duces to a distance metric equal to the trace norm of pair
wise differences between outerproducts of the signal matri
ces.
Cutoff rate analysis has frequently been adopted to es
tablish practical coding limits [7,2] as the cutoff rate spec
This work was performed, in part, while the first author was visiting
the Mathematical Sciences Research Center.
A lower bound is given on the largest possible minimum
distance for arbitrary sets of signal matrices of fixed finite
0780363396/00/$10.00 02000 IEEE
168
Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 15:03 from IEEE Xplore. Restrictions apply.
Page 2
dimension. This is also a lower bound on the maximum
distance of signals in the optimal cutoff rate attaining signal
set.
A necessary and sufficient condition for a signal proba
bility distribution to attain cutoff rate is that it equalize the
decoder error rate over all possible signal matrices. We call
this the equalization condition and it plays a central role in
this work.
The determination of the K dimensional cutoff rate re
duces to maximization of a quadratic form over the set of
feasible constellations, defined as those constellations which
satisfy both the peak power constraint and the finite dimen
sional equalization condition. This quadratic form is simi
lar to that arising in the Capon/MVDR method for adaptive
beamforming arrays. If the feasible set of K dimensional
constellations is empty then the optimal constellation is nec
essarily of dimension less than K.
For low symbolrate the optimal constellation is a set
of scaled mutually orthogonal unitary matrices in 67 *' M .
This constellation also maximizes minimum distance over
all constellations of the same dimension. When SNR is low
the rank of the signal matrices in the constellation is one
and cutoff rate is achieved by applying all power to a single
antenna element at a time. As the SNR increases the rank
of the signal matrices increases and more and more antenna
elements are utilized. Interestingly, the number of receive
antennas N plays no role whatsoever in determining how
many transmit antennas should be used.
trace norm of pairwise differences between outerproducts
of the codeword matrices.
We end this subsection with a result that parallels Theo
rems 1 and 2 of Marzetta and Hochwald [5], but covers the
case of peak power constrained signal sets.
Proposition 1 Assume that the transmitted signal S is con
strained to satisfy the peak power constraint llS1(2 < MT.
The peak power constrained cutoff rate attained with M >
T transmit antennas is the same as that attained with M =
T antennas. Therefore, there is no advantage to using more
than T transmit antennas. Furthermore, for M 5 T the
signal matrix which achieves peak power constrained cut
off rate can be expressed as S = VA where V is a T x T
unitary matrix, A = [AM, 0IT is a T x M matrix, and AM
is a diagonal M x M matrix.
Readers familiar with Theorems 1 and 2 of [5] might
suspect that characterization of the statistical distribution P
of the optimal cutoff achieving signal matrix S can be ob
tained. Indeed, paralleling the arguments of [5], it can be
shown that, as tr{SSH} 5 TM is invariant to unitary pre
multiplication of S and as the maximization in the definition
of R, is over a concave function of P, the peakpower con
strained cutoff rate is attained by random matrices of the
form S = VA where V is a T x T isotropically distributed
matrix, A = [AM, 0IT is a random T x M diagonal matrix,
and V and A are statistically independent.
Define a constellation as follows
2. CUTOFF RATE REPRESENTATIONS
We first obtain an integral representation for the cutoff rate
R, which depends on a pairwise dissimilarity measure D(SillSj)
over the set of codewords S:
R, = m u  1x1 LIES
PEP
Definition 1 A set of matrices {&}El in
word constellation if all assigned codeword probabilities P i
are strictly positive, i = 1, . . . , K.
is a code
In the following sections we specialize to the case of dis
dp(s2) ,No(& 1l.S~). (2~rete signal constellations for which equalizer distributions
are always optimal.
S2ES
dP(S1) 1
where
3. DISCRETE CONSTELLATIONS
D(S1 IlS2)
4 In
rn (2) the max.imization is performed Over a suitably con
strained set p of probability distributions p defined over
the set of peak constrained codewords S.
The K dimensional cutoff rate, defined as the cutoff rate
for constellations whose dimension does not exceed K, is
the appropriate limiting factor for practical coding schemes.
Define the feasibility set Ssak of Kdimensional con
stellations
SEak = {{&}El
EG1LK E R : ,
The dissimilarity measure D(Si IlSj) is a decreasing func
tion of the spatial correlation between prewhitened versions
of pairs of codeword matrices. It can be shown that D(S1IlSz) =
772/SllSlS~S2S,HI12+o(772),
so that for low SNR the dis
similarity measure reduces to a distance metric equal to the
: si E @ T M , llSi112 <_ TM,
SiSy # S j S : , i # j } . (4)
169
Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 15:03 from IEEE Xplore. Restrictions apply.
Page 3
TM n
whereiK = [I,. . . , 1IT E RK andEK = ( (eND(SillSj)
is the K x K dissimilarity matrix of the constellation. EK is
positive definite as long as the outerproduct matrices {SiSF}zl
are distinct. It can be shown that SEak is the set of “equal
izer” constellations for which there exists an “equalizer prob
ability vector” PK satisfying
))i,j=I
EKPK = CIK
for some c > 0.
The following representation theorem asserts that Ro(K)
is attained by an equalizer constellation which maximizes a
simple quadratic form.
Proposition 2 Let K be a positive integel: The peak power
constrained K dimensional cutoff rate R, (K)_is attained
by a constellation in one of the feasible sets Speak, k =
1,. . . , K, and
Observe that by taking K = 00 in Proposition 2, we ob
tain the cutoff rate of constellations of countable, but pos
sibly infinite, dimension. The objective function lzEilfk
maximized in (5) is similar to the criterion used in Capon’s
method, also known as minimum variance distortionless re
sponse (MVDR), for adapting the weights of a beamform
ing array of antenna elements and for high resolution spec
tral estimation [4].
4. SPECIAL LIMITING CASES
Here we specialize the cutoff rate to several limiting regimes.
4.1. Large Dimension K
Recall the definition Dmin = minifj D(Si1lSj). When
K 5 [T/MJ we will see (Proposition 5) that a set of sig
nal matrices {&}El exists for which D(SillSj) = Dmin
for all i # j , and which simultaneously attains the cutoff
rate Ro(
K) and attains the maximum possible value Dmin.
The following result establishes a lower bound on the largest

possible value Dzn = max{Si)~l:SiEspKcak
A constellation { Si}El achieving Dmin = Dzn is called
a maximin constellation of dimension K.
Figure 1: For gT(2’l  1 ) < K 5 gT(2’  1) and p
a positive integer the vectors of singular values of the con
stellation are optimal (maximum Dmin) when they form a
uniform lattice and there are exactly K = g~(2P  1) sig
nals in the constellation. In the figure T = 2 and p = 3.
3.
{(il,. . . , i ~ )
fine the integer valued function
: il E (0, 1/(2p  1),2/(2P  l), . . . , 1)). De
For example, 949) = (q + l)(q + 2)/2 and ~ ( 9 )
l)(q+2)(q+3)/6. gT(2’1) is the number of lattice points
of C p , ~
which are inside of the Tdimensional unit simplex
{ (uI,.
. . , U T ) :
= (q +
T u1 5 1, UI E [0, l]} (See Fig. 1).
Proposition 3 For given K > 1 letp be the unique integer
for which gT(2’l  1 ) < K 5 g~(2P  1). Then
Thejrst inequality is tight when K = g~(2P  1) and the
antidiagonal matrices Zi = SiSF  diag(SiSF) satisfy
llZi  Zjll = 0, i # j.
The proposition says that, for sufficiently small SNR
mini+j ~ ( ~ ~ 1 1 s ~ ) .
q2, Dzin cannot asymptotically decrease to zero faster than
rate K2/T. This asymptotic rate can be achieved, for ex
ample, in the case that M = T, the { S i S ~ } ~ = l ’ ~
agonal and form the constellation of a linearly constrained

be the uniform lattice of
2PT points covering the Tdimensional unit cube: C p , ~
=
~~
are di
For positive integer p let C p , ~
lattice code [ 11 in RT. Specifically, each S i is a square ma
trix with orthogonal rows and the vectors of diagonals of the
170
Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 15:03 from IEEE Xplore. Restrictions apply.
Page 4
SiSF matrices are locations of the tie points of the K point
lattice shown in Fig. 1.
The above result leads to a lower bound on the average
distance D(S; IIS;)
of the optimal Kdimensional cutoff
rate achieving constellation, {S;}El, where
We can show [3] that D(STIlSj’) 2 DS,, so that
4.2. Low Dimensional Constellations
When the number of signal matrices K to be considered
is sufficiently small significant simplification of the cutoff
rate computation is possible. In particular, one obtains opti
mality of a set of scaled mutually orthogonal unitary signal
matrices and a simple form form for Ro(K).
The first result specifies the solution to optimization of
the dissimilarity measure D(SillSj) defined in (3).
For given q, T and M define the integer MO
We will see below that under some conditions MO is the
rank of the signal matrices Si in the optimal Kdimensional
constellation.
The proof of the following proposition is based on the
alternative but equivalent representation for D(S1 IIS2)
where IC is a M x M multiple signal correlation matrix
K, = SfSl
Proposition 4 Let 2M 5 T. Then
where, for j = 1,2,
The assumption 2M 5 T is critical and ensures that the
singular vectors of SI and 5’2 can be chosen as mutually
orthogonal for any set of singular values.
The rank MO of the optimal matrices S1 and S 2 in
creases from 1 to M as the SNR parameter qTM increases
from 0 to 00 (see Fig. 2). Numerical evaluation has shown
that the functional relationship between MO and S N R is well
approximated by the relation
(1, LaqTM + b + 0.481)
MO x m a
where a, b are the slope and intercept of the least squares
linear fit to the function y(z) = argma~,,~,~,,,,m ln[(l +
~ / ( 2 m ) ) ~ / (
1 +z/m)]. The approximation is a lower bound
and underestimates the exact value of MO, given by (7), by
at most 1 over less than 0.5% of the SNR range shown in
Fig. 2 (0 < qTM 5 120). If the SNR is sufficiently large,
e.g. (from Fig. 2) qTM 2 17 for M = 6 and T 2 12,
MO = M and the optimal signal matrices utilize all M
transmit antennas. On the other hand for small SNR, i.e.
(from Fig. 2) qTM < 4, MO = 1 and the optimal sig
nal matrices apply all available transmit power to a single
antenna element over the coherent fade interval T.
The final result of this section is an expression for the
cutoff rate.
Proposition 5 Let 2M 5 T and let MO be as defined in
(7). Suppose that MO 5 min{M,T/K}. Then the peak
constrained K dimensional cutofSrate is
and D,,
stellation attaining R, (K) is the set of K rank MO mutually
orthogonal unitary matrices and the optimal probability as
signment is uniform: Pi* = 1/K, i = 1,. . . , K.
is given by (8). Furthermore, the optimal con
Any unitary transformation on the columns (spatial co
D,,, def max
s 1 ,S2ESpKelk
D( Sl IlS2) = MO In (l + qTM/(2Mo))‘ ordinates) of a set of signal matrices produces a set of sig
‘nal matrices with identical Dmin. In particular, any set of
K mutually orthogonal T x MO permutation matrices has
optimal distance properties. This simple set of signal matri
Furthemore, the optimal signal matrices which attain
D,,, can be taken as scaled rank MO
mutually orthogonal
unitary T x M matrices of the form
element at a time, among a total of MO 5 M elements, in
each of the available T time slots. Since R,(K) is increas
ing in K the maximum cutoff rate achievable using these
1 + VTM/MO
ces corresponds to transmitting energy on a single antenna
SI = JmpiEcP1,
s 2 = J m p i E i E 2
171
Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 15:03 from IEEE Xplore. Restrictions apply.
Page 5
mutually orthogonal unitary matrices is obtained by using
the maximum possible number of them: K = LT/M,J.
Observe that the resulting optimal constellation may corre
spond to a code of quite.10~ symbol rate, e.g. for MO =
M = T/2 the symbol rate is only 1 bitpersymbol.
1988.
and Application, PrenticeHall, EnglewoodCliffs N.J.,
It is noteworthy that the optimal peak constrained signal
constellation specified by Proposition 5 does not include the
zero valued signal matrix Si = 0. Including zero in the sig
nal constellation would allow signalling using onoff key
ing. Onoff keying is often proposed for average power con
strained signalling over low SNR channels since it permits
energy discrimination at the receiver. As contrasted with on
off keying all signals in the optimal peak constrained signal
set have equal power. We conjecture that the zero signal
would result from replacing the peak power constraint with
an average power constraint in Proposition 5.
[5] T. L. Marzetta and B. M. Hochwald, “Capacity of a mo
bile multipleantenna communication link in Rayleigh
fading,” IEEE Trans. on Inform. Theory, vol. IT45, pp.
139158. Jan. 1999.
[6] J. E. Savage, “Sequential decoding  the computation
problem,” Bell Syst. Tech. Journ., vol. 45, pp. 149175,
Jan. 1966.
[7] EQ. Wang and D. J. Costello, “Probabilistic construc
tion of large constraint length trellis codes for sequen
tial decoding,’’ IEEE Trans. on Communications, vol.
COM43, no. 9, pp. 24392448, Sept. 1995.
[8] J. M. Wozencraft and R. S. Kennedy, “Modulation and
demodulation for probabilistic coding,” IEEE Trans. on
Inform. Theory, vol. IT12, pp. 291297, July 1966.
4.3. Conclusions
We have derived representations for the computational cut
off rate for space time coding under the Rayleigh flat fading
channel model under a peak transmitted power constraint.
For finite dimensional constellations the cutoff rate and the
optimal signal distribution were specified as a solution to a
quadratic optimization problem and it was shown that op
timal constellations have codeword distributions which sat
isfy an equalization condition. This characterization of opti
mality motivated us to study properties of the set of feasible
constellations which satsify the equalization property. Eas
ily verifiable necessary and sufficient conditions were given
for validating that a given signal constellation lies in the fea
sible set. Based on one of these conditions in [3] a greedy
procedure was proposed for recursively constructing or re
fining a good feasible constellation.
5. REFERENCES
[I] J. H. Conway and N. J. A. Sloane, Sphere packings,
lattices and groups, SpringerVerlag, New York, NY,
1988.
10
‘0
20 40
60
80
100
120
81
I
[2] J. convolutional Hagenauer, codes,”
vol. IT42, no. 2, pp. 429445, Mar. 1996.
“Iterative
IEEE
decoding
Trans. of on binary
Inform. block
Theory,
and “ L <  ; d = = ;  = l
2
0
5
10
SNR
15
20
[3] A. 0. Hero and T. L. Marzetta, “On computational cut
off rate for space time coding,” Technical report, Bell
Laboratories, Lucent Technologies, Murray Hill, NJ,
2000.
Figure 2: Top panel shows MO given by (7) as a function of
the SNR parameter $ ! ‘ A l . Bottom panel is blow up of first
panel over a reduced range of SNR.
[4] S. M. Kay, Modern Spectral Estimation: Theory
172
Authorized licensed use limited to: University of Michigan Library. Downloaded on May 6, 2009 at 15:03 from IEEE Xplore. Restrictions apply.