ArticlePDF Available

Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise

Authors:

Abstract and Figures

We report on the feasibility of k-means clustering techniques for the dynamic threshold detection of encoded q-ary symbols transmitted over a noisy channel with partially unknown channel parameters. We first assess the performance of k-means clustering technique without dedicated constrained coding. We apply constrained codes which allows a wider range of channel uncertainties so improving the detection reliability.
Content may be subject to copyright.
Received January 6, 2020, accepted January 20, 2020, date of publication January 23, 2020, date of current version January 30, 2020.
Digital Object Identifier 10.1109/ACCESS.2020.2968945
Dynamic Threshold Detection Using Clustering
in the Presence of Channel Mismatch
and Additive Noise
KEES A. SCHOUHAMER IMMINK 1, (Fellow, IEEE), AND KUI CAI 2, (Senior Member, IEEE)
1Turing Machines Inc., 3016 Rotterdam, The Netherlands
2Department of Science, Singapore University of Technology and Design (SUTD), Singapore 487372
Corresponding author: Kees A. Schouhamer Immink (immink@turing-machines.com)
This work was supported in part by the Singapore University of Technology and Design (SUTD) Start-Up Research Grant (SRG) under
Grant SRLS15095, in part by the Singapore Ministry of Education Academic Research Fund Tier 2 under Grant MOE2016-T2-2-054, and
in part by the RIE2020 Advanced Manufacturing and Engineering (AME) Programmatic under Grant A18A6b0057.
ABSTRACT We report on the feasibility of k-means clustering techniques for the dynamic threshold
detection of encoded q-ary symbols transmitted over a noisy channel with partially unknown channel
parameters. We first assess the performance of k-means clustering technique without dedicated constrained
coding. We apply constrained codes which allows a wider range of channel uncertainties so improving the
detection reliability.
INDEX TERMS Data storage systems, non-volatile memories, channel mismatch, k-means clustering,
dynamic threshold detection.
I. INTRODUCTION
In data storage products, user data are written into physical
attributes that can be either magnetic [1], electronic [2], opti-
cal [3], or even biological such as DNA [4]. Due to inevitable
process variations, called noise, the magnitude of the physical
attributes may deviate from their nominal values. In addition,
we have a long-term instability, called drift [5], which may
lead over time to changes of the physical attributes. As the
detector is ignorant of the long-term drift of the physical
attributes, this leads to a phenomenon called channel mis-
match. Channel mismatch without dynamic detection may
lead to significant degradation of the error performance,
as shown, for example, in [6]. As not all code blocks in a
random-access memory are written at the same time, the drift
at read time, which is a function of the lapsed time since
writing the data blocks, may therefore vary per code block.
While process variations may vary from symbol to symbol,
it is assumed here that the drift term is constant within a block
of symbols, but may vary from code block to code block.
The changes in drift per block basis precludes the usage of
conventional dynamic drift estimation, which is based on pre-
viously retrieved codewords. As a consequence, the detector,
ignorant of the actual drift, must therefore estimate the drift
The associate editor coordinating the review of this manuscript and
approving it for publication was Wen-Long Chin .
on a block of symbols basis, and adjust its detector parameters
for optimizing its performance.
The detector resilience to unknown mismatch by drift can
be improved in various ways, for example, by employing
coding techniques. Balanced codes [6]–[9] and composition
check codes [10], [11], in conjunction with Slepian’s optimal
detection [12] offer excellent resilience in the face of channel
mismatch on a block of symbols basis. These coding and sig-
nal processing techniques are often considered too expensive
in terms of code redundancy and hardware, in particular when
high-speed applications are considered.
Recently, Pearson distance detectors have been advocated
as they are immune to mismatch on a block of symbols
basis [6]. For the binary case, q=2, the redundancy is attrac-
tively low and the complexity of the Pearson detector scales
with n. Neural network-based dynamic threshold detection
for non-volatile memories have been presented in [5]. For
larger alphabet size, q, the number of arithmetical operations
of grows exponentially with the codeword length, n, so that
these methods become an impracticability [13]. Alternative
detection methods for larger qand nthat are less costly in
resources are welcome.
We investigate new detection techniques for q-ary data,
q>2, that are conveyed over channels distorted by addi-
tive noise and unknown drift of the channel parameters.
The new detection techniques are based on k-means cluster
VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ 19319
K. A. S. Immink, K. Cai: Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise
analysis [14]–[16] that classifies the nreceived symbols
into qclusters. Detection with given initial estimates of
the (unknown) channel parameters iterates to improved esti-
mates of the sent n-symbol codeword. In this work, we focus
on the application of the clustering technique for signal detec-
tion for non-volatile memory (NVM) technologies, which
typically have microsecond to nanosecond read access time,
and ultra-low power consumption. We choose to use the
k-means clustering algorithm to illustrate our ideas, since it
is simple and effective, and can satisfy the high-speed and
low-complexity requirement of NVMs.
Section II commences with preliminaries and a description
of the channel model. Prior art detection schemes are dis-
cussed in Section III. In Section IV, we present a new detec-
tion technique based on k-means clustering, and the error
performance of the prior art and new schemes are assessed.
In Sections Vand VI, we adopt a simple linear channel model
for improving the detection quality.
II. PRELIMINARIES AND CHANNEL MODEL
Let SQnbe a codebook of n-symbol codewords x=
(x1,x2,...,xn) over the q-ary alphabet Q= {0,...,q1}.
The codeword, xS, is translated into physical attributes,
where the logical symbols, xi=kQ, are written at a
nominal (physical) level k. Due to signal dependent drift,
which depends on the original physical level k, the physical
level is at read time equal to k+bk, where the drift term
bkRdenotes the average deviation from the nominal
value k. The qdrift parameters bkare average deviations from
the nominal levels, and they are assumed to be relatively small
with respect to the (normalized) unity difference between
neighboring signal levels. Besides the distortion by unknown
drift, the received symbols are distorted by unknown additive
noise caused by random telegraph noise [17], [18] and fabri-
cation process variations during the write operation. We are
now in a position to write down the channel model.
Let the codeword, x, consisting of n q-ary symbols be
sent or stored. The symbols, ri, of the retrieved vector r=
(r1,...,rn) are distorted by unknown additive noise and
signal dependent drift terms, and given by
ri=xi+bxi+νi,1in.(1)
The received word, r, is corrupted by (assumed) additive
Gaussian noise ν=(ν1, . . . , νn), where νiRare zero-mean
independent and identically distributed (i.i.d) noise samples
with normal distribution N(0, σ 2). The quantity σ2R
denotes the noise variance. Since the qsignal dependent drift
terms, bxi=bk,xi=kQ, are fixed for an n-symbol word,
and may vary per block of nsymbols, previously received
n-symbol blocks cannot be used to estimate the unknown drift
terms. Thus, the estimation of the qreal-valued bks and the n
integer-valued xis must be accomplished having knowledge
of the nobserved ri’s, see (1), plus the assumptions on the
distributions of the noise νiand signal dependent drift bk,
only.
Unambiguous solution of (1) is not possible for large val-
ues of the signal dependent drift term bk, and restrictions must
be made on its range. For unambiguous detection we must
avoid overlap between the physical level, k+bk, associated
with the logical symbol kQ, and its neighboring physical
level k+1+bk+1associated with the logical symbol k+1.
In an extreme example, we may have b0=1 and b1= −1,
so that, as a result, the logical values of the ‘0’s and ‘1’s and
their representations by the physical levels ‘0’s and ‘1’s are
swapped, which should be avoided. Such undesired situation
is avoided if we stipulate that
b0<1+b1<2+b2<··· <q1+bq1(2)
or, we have found the restrictions
bk1bk<1,1kq1.(3)
We first design, in Section IV, a detector for the above
model (1), where the qunknown offsets, bks, are uncorre-
lated. Thereafter, in Sections Vand VI, we distinguish two
special cases, where the qunknown bks are correlated. For
the first model, we assume
bk=(a1)k+b,(4)
where ais an unknown attenuation, or gain, of the channel,
and bis an unknown offset,aand bR. We simply find
using (1)
ri=axi+b+νi.(5)
Note that for the binary case, q=2, we can always rewrite (1)
by (5), substituting b=b0and a=1+b1b0. The
condition (3) implies a>0. For the second model, called
offset-only model, a=1, all qdrift terms, bk, are equal,
so that we simply obtain ri=xi+b+νi.
We start with a brief discussion of prior art detection
methods.
III. PRIOR ART DETECTION SCHEMES
We discuss three prior art detection schemes and their relevant
attributes.
A. FIXED THRESHOLD DETECTION (FTD)
A conventional fixed threshold detector (FTD), also called
symbol-by-symbol detector, straightforwardly quantizes the
nreal-valued ris to an integer, ˆxiQ. Let the threshold
function be denoted by ˆxi=8ϑ(ri), ˆxiQ, where the
threshold vector ϑ=(ϑ0, . . . , ϑq2) has q1 (real) elements,
called thresholds or threshold levels. The threshold vector
satisfies the order
ϑ0< ϑ1< ϑ2<··· < ϑq2.(6)
The quantization function, 8ϑ(u), of the threshold detector is
defined by
8ϑ(u)=
0,u< ϑ0,
i, ϑi1u< ϑi,1iq2,
q1,uϑq2.
(7)
19320 VOLUME 8, 2020
K. A. S. Immink, K. Cai: Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise
For a fixed threshold detector the q1 detection thresholds
values, ϑi, are equidistant at the threshold levels
ϑi=1
2+i,0iq2.(8)
Threshold detection has been applied for its implementation
simplicity. However, we may pay the price for its simplicity
elsewhere as the error performance seriously degrades in the
face of channel mismatch [6]. A detector that dynamically
adjusts the threshold vector, ϑ, is a necessity that offers solace
in the face of channel mismatch. A typical example of such a
detector is described in the next subsection.
B. DYNAMIC THRESHOLD DETECTION (MIN-MAX
DETECTOR)
We assume that the channel model is, see (5), ri=axi+b+νi,
where the gain, a>0, and offset, b, are unknown parameters,
except for the sign of a. In case S=Qn, that is all possible
codewords are allowed, mismatch immune detection on a
block of symbols basis is not possible since such a detector
cannot distinguish between the word ˆ
xand its shifted and
scaled version ˆ
y=c1ˆ
x+c2. A designer must judiciously
select codewords from Qngiven adequate constraints that
enables mismatch-immune detection. For example, we select
for Sthose codewords where the symbols ‘0’ and ‘q1’
must be both at least once present. For the binary case, q=2,
this implies a slight redundancy as only the all-‘0’ and all-‘1’
words have to be barred, see Subsection VI-A for details.
Then, the detector can straightforwardly estimate the gain and
offset by
ˆa=maxiriminiri
q1(9)
and
ˆ
b=min
iri,(10)
where ˆaand ˆ
bdenote the estimates of the actual channel gain
and offset. The dynamic thresholds, denoted by ˆ
ϑi, are scaled
in a similar fashion as the received codeword, that is,
ˆ
ϑi= ˆaϑi+ˆ
b,0iq2.(11)
It has been shown in [19] that the min-max detector operates
satisfactorily over a large range of unknown parameters a
and b. However, since ˆaand ˆ
bare biased estimates of aand b,
the above dynamic threshold detector loses error performance
with respect to the ideal matched case, especially for larger
codeword length n. The detector complexity scales linearly
with nas its principal cost is the finding of the maximum and
minimum of the nreceived symbol values using (9) and (10).
Alternatively, detection based on the prior art Pearson dis-
tance, discussed in the next subsection, improves the error
performance, but with mounting hardware requirements.
C. PEARSON DISTANCE DETECTION
Immink and Weber [6] advocated the Pearson distance
instead of the conventional Euclidean distance for improv-
ing the error performance of a mismatched noisy channel.
We first define two quantities, namely the vector average of
the n-vector z
z=1
n
n
X
i=1
zi(12)
and the (unnormalized) vector variance of z
σ2
z=
n
X
i=1
(ziz)2.(13)
The Pearson distance, δp(r,ˆ
x), between the received vector r
and a codeword ˆ
xSis defined by
δp(r,ˆ
x)=1ρr,ˆ
x,(14)
where
ρr,ˆ
x=Pn
i=1(rir)(ˆxi− ˆx)
σrσˆx
(15)
is the (Pearson) correlation coefficient. It is assumed that both
codewords xand ˆ
xare taken from a judiciously chosen code-
book S, whose properties are explained in subsection VI-A.
A detector based on the Pearson distance outputs the
codeword
xo=argmin
ˆ
xS
δp(r,ˆ
x).(16)
It can easily be verified that the minimization of δp(r,ˆ
x), and
thus xo, is independent of both aand b, so that the detection
quality is immune to unknown drift of the quantities aand
bon a block of nsymbols basis. The minimization opera-
tion (16) requires |S|computations, which is impractical for
larger S. The number of computations can be reduced to K,
the number of constant composition codes that constitutes the
codebook S, given by [6]
K=n+q3
q1.(17)
For the binary case, q=2, we have K=n1, so that
the detection algorithm (16) scales linearly with n. For the
non-binary case, it is hard to compute or simulate the error
performance of minimum Pearson distance detection in a
relevant range of qand nas the number, K, of operations
grows rapidly with both qand n.
The three prior art detection methods discussed above
have drawbacks in error performance and/or complexity, and
to alleviate these drawbacks, viable alternatives are sought
for. In the next section, we propose and investigate a novel
detection method with less complexity requirements, which
is based on clustering techniques.
IV. DATA DETECTION USING K-MEANS CLUSTERING
In the next subsection we describe the basic k-means clus-
tering algorithm, and present results of simulations for the
unmatched noisy channel.
VOLUME 8, 2020 19321
K. A. S. Immink, K. Cai: Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise
A. BASIC K-MEANS CLUSTERING ALGORITHM
We assume that a cluster is associated with one of the
q,q>2, symbol values. The k-means clustering technique,
see for example [15], aims to partition the nreceived symbols
riinto k=qclusters (sets), V= {V0,V1,...,Vk1}
by minimizing the within-cluster sum of squares, which is
defined by
argmin
V
k1
X
i=0X
rjVi
(rjµi)2,(18)
where the centroid µiis the mean of the received symbols in
cluster Vi, or
µi=1
|Vi|X
rjVi
rj.(19)
The binary case, q=2, has been studied in the context of
symbol timing recovery by Zhao et al. [14], where the two
pulse amplitudes are unknown.
The k-means clustering algorithm is an iteration routine
that finds a solution of (18). At the start of the iteration
algorithm, the sets V(1)
i, 0 ik1, are void. The
superscript integer in parentheses, (t),t=1,2, . . ., denotes
the iteration count. In addition, we initialize the kcentroids
µ(1)
i, 0 ik1, by a ‘reasonable’ choice. The choice
of the initial centroids has a bearing on word error rate and
the speed of convergence of the iteration process. Forgy’s
method [20], for example, a well-known method described in
the literature, randomly chooses ksymbols (assuming k<n),
ri, from the n-symbol received vector r, and sets these as
the initial centroids µ(1)
i, 1 ik. Forgy’s approach is
not followed here as we developed improved initial centroids
aiming to reduce the number of iterations and the word error
rate.
Let the order of the centroids be given by
µ(t)
0< µ(t)
1<··· < µ(t)
q1.(20)
We iterate the next two steps until the symbol assignments no
longer change and the iteration routine has come to a halt.
Assignment step: The n ri’s, 1 in, are assigned to
the ksets V(t+1)
jas follows. If riis closest to the centroid
µ(t)
`, or
`=argmin
jriµ(t)
j2
,(21)
then riis assigned to cluster V(t+1)
`. The elements of
the (intermediate) decoded codeword, denoted by ˆ
x(t)=
(ˆx(t)
1,...,ˆx(t)
n) is found by
ˆx(t)
i=φV(t)(ri),1in,(22)
where φV(t)(ri)=jsuch that riV(t)
j.
Updating step: The updated means µ(t+1)
j,jQis
found by
µ(t+1)
j=1
|V(t+1)
j|X
riV(t+1)
j
ri,jQ.(23)
If |V(t+1)
j| = 0 then we set µ(t+1)
j=µ(t)
j(that is,
no update).
We run the above algorithm until the intermediate decoded
word is unchanged at iteration step, t=to, that is, we have
ˆ
x(to1) =ˆ
x(to). Then the final estimate of the sent codeword is
xo=ˆ
x(to). The k-means cluster algorithm always converges
to a simple steady state, and limit cycles do not occur [21].
Note that the process may reach a local minimum of the
within-cluster sum of squares (18).
B. ASSIGNMENT STEP
Let us focus on the assignment step of the k-means clustering
technique given by (21). Considering the order (20) of the
centroids µ(t)
j, we conclude that the symbol rilies between,
say, µ(t)
uriµ(t)
u+1, 0 uq2. Thus, we have
`=argmin
j∈{u,u+1}riµ(t)
j2
.
As
riµ(t)
u2riµ(t)
u+12
=2riµ(t)
u+1µ(t)
u+µ(t)
u2µ(t)
u+12
=µ(t)
u+1µ(t)
u2riµ(t)
uµ(t)
u+1,(24)
we obtain
`=
u,ri<µ(t)
u+1+µ(t)
u
2,
u+1,ri>µ(t)
u+1+µ(t)
u
2.
(25)
Using (7), we yield
`=argmin
jriµ(t)
j2=8ˆ
ϑ(ri),(26)
where
ˆ
ϑi=µ(t)
i+1+µ(t)
i
2,0iq2.(27)
The vector, ˆ
x(t), equals
ˆx(t)
i=8ˆ
ϑ(ri),1in.(28)
From the above, we see that the k-means clustering detec-
tion method, following object function (18), is a dynamic
threshold detector, where the threshold vector, ˆ
ϑ, is iteratively
updated with the means of the members of each cluster
using (23).
C. SIMULATIONS RESULTS
We now investigate the error performance of the channel
based on model (1). The qunknown drift parameters, bk,
kQ, are assumed to be taken from a zero-mean continuous
uniform distribution with variance σ2
b. Clearly, the parameters
bk’s are within the range 3σbbk3σb.
At the beginning of the iteration detection process, the cen-
troids are set to µ(1)
i=i,iQ, and the decoder iterates the
19322 VOLUME 8, 2020
K. A. S. Immink, K. Cai: Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise
FIGURE 1. Curve (a’): WER of fixed threshold detection (FTD), and curve
(b’): k-means clustering detection versus SNR = −20 log σ(dB) for n=64,
q=4, and σb=0.1. Curves (a) and (b) show the same parameters, but
σb=0. Curve (c) shows the upperbound (29) to the word error rate of a
fixed threshold detector for an ideal noisy channel, q=4 and n=64.
assignment and updates following the steps discussed above.
Figure 1shows results of simulations for the case n=64
and q=4. We compare the word error rate (WER) of
traditional fixed threshold detection and the newly developed
dynamic threshold detection based on k-means clustering
versus signal-to-noise (SNR). The SNR is defined by =
20 log σ. We display two cases of interest, namely σb=0
(no variations, ideal channel) and σb=0.1 (random drift
parameters). We further display the upper bound of the word
error rate of a threshold detector for the ideal additive noise
channel [6]
WER <2(q1)
qnQ 1
2σ.(29)
The error performance of k-means clustering detection for
σb=0 is close to the performance of both theory and
simulation practice of conventional fixed threshold detection.
We infer that for σb=0.1, k-means clustering detection is
superior to fixed threshold detection.
An important (time) complexity issue is the number of
iterations, which depends on the integers q,n, and the signal-
to-noise ratio, SNR. Although the convergence of the iteration
process is guaranteed [21], the speed of convergence, and thus
the complexity issue is an open question that we investigated
by simulations.
Results of simulations for n=64, q=4, and σb=0.1 are
shown Table 1(same parameters as used in the simulations
depicted in Figure 1). At an SNR =17 dB, around 91%
TABLE 1. Histogram of the number of iterations for q=4, n=64, and
σb=0.1.
of the received words is detected without further iterations.
In 8% of the detected words, only a single iteration of the
threshold levels is required before finishing the iteration
process. Essentially no iterations are required at a higher
SNR =20 dB. As no iterations are needed in the large
majority of cases,we infer that k-means clustering detection
outperforms fixed threshold detection at the cost of only a
slight additional (time) complexity.
The situation is, however, slightly more subtle than can be
inferred from perusing Figure 1as a detector flaw is hidden
behind the average values. We investigate this phenomenon
for the binary, q=2, case. If we look closer to the error
components, it turns out that codewords with a small or large
weight, where the weight of a codeword is the sum of the
1’s in that codeword, are much more prone to error than
codewords with (almost) equal numbers of ones and zeros.
Figure 2, Curve (a), shows for the k-means clustering method
the WER as a function of the weight of the sent codeword,
n=12, for a binary, q=2, ideal channel, σb=0. We infer
that words with a low or high weight (except the all-one and
all-zero words) are more prone to error than words with a
weight close to n/2, a so-called balanced word. The error
performance of fixed threshold detection is independent of
the weight of the sent codeword, see Figure 2, Curve (b).
FIGURE 2. WER of k-means clustering detection, curve (a), and fixed
threshold detection (FTD), curve (b), versus the weight of the sent
codeword at an SNR = 17 dB for n=12, q=2, and σb=0 (ideal channel).
V. UNKNOWN GAIN AND OFFSET (SMALL RANGE OF
UNCERTAINTY)
In this section, we assume that the channel model, see (5),
ri=axi+b+νi,
applies. If the tolerance range of the gain, a, is close to
unity and the tolerance range of the offset, b, is close to
zero, we may directly apply the basic k-means clustering as
outlined in the previous section. Both aand bmust lie so close
to their nominal values that a fixed threshold detector oper-
ates correctly in the noiseless case. Then, the initialization
VOLUME 8, 2020 19323
K. A. S. Immink, K. Cai: Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise
step, using the fixed threshold detector, furnishes sufficiently
reliable data for the iterations to follow. From the definition
of a fixed threshold decoder, see (7), we simply derive the fol-
lowing tolerance ranges of aand bthat guarantee a flawlessly
operating threshold detector, namely
b< ϑ0=1
2,
ϑi1<ai +b< ϑi,1iq2,
a(q1) +b> ϑq2=q3
2,(30)
or
b<1
2,
i1
2<ai +b<i+1
2,1iq2,
a(q1) +b>q3
2.(31)
Results of computer simulations are displayed in Figure 3,
where for the case n=64 and q=4, the WER of fixed
threshold detection and detection based on k-means cluster-
ing versus SNR are compared. The channel gain is assumed
to be a=0.95 and b=0, respectively. We conclude that
the cluster detector shows a greater resilience in the face of
unknown gain, a, and additive noise than the fixed threshold
detector, where the parameters aand bare assumed to have
a limited range of uncertainty. In case, however, they have
a wider tolerance range than prescribed by (31), it is not
possible to unambiguously detect the codeword with a fixed
threshold detector. Then, the detector needs assistance, and
constrained coding is applied to assist in overcoming this
difficulty as discussed in the next section.
FIGURE 3. WER of both k-means clustering detection (cluster) and fixed
threshold detection versus SNR for n=64, q=4, a=0.95, and b=0. The
upperbound to the word error rate of a fixed threshold detector for the
ideal noisy channel given by (29) is shown as a reference.
VI. UNKNOWN GAIN AND OFFSET (LARGE RANGE OF
UNCERTAINTY)
In this section, we focus on the situation where we antici-
pate that in (5) both parameters aand bhave such a great
range of possible values that a fixed threshold detector fails
in the majority of cases. In the next subsection, we show,
by example, that in such a case it is impossible to distinguish
between certain nettlesome situations, and constrained cod-
ing becomes a requirement to solve the ambiguity.
A. CONSTRAINED CODING
In order to cope with larger uncertainties of both parameters
aand b, we face an ambiguity problem. For example, let
q=5, and let (2,4,4) be the received vector. Clearly, it is
impossible to distinguish between the two choices, where the
sent codeword is (2,4,4) and a=1 or where (1,2,2) and
a=2. Let Sbe the adopted codebook, then we can cope
with the above ambiguity if (2,4,4) Sthen (1,2,2) /S,
or vice versa. The name Pearson code was coined [19] for a
set of codewords that can be uniquely decoded by a detector
immune to large uncertainties in both gain a,a>0, and
word offset b. Pearson code design can be found in [22].
Codewords in a Pearson code, S, satisfy two conditions,
namely
Property A: If xSthen c1+c2x/Sfor all c1,c2R
with (c1,c2)6= (0,1) and c2>0.
Property B: x=(c,c,...,c)/Sfor all cR.
We adopt a Pearson code that has codewords with at least
one ‘0’ symbol and at least one ‘q1’ symbol. We may easily
verify that such codewords satisfy Properties A and B. The
number of allowable n-symbol codewords equals [19]
|S| = qn2(q1)n+(q2)n,q>1.(32)
For the binary case, q=2, we simply find that
|S| = 2n2
(both the all-‘1’ and all-‘0’ words are deleted).
B. REVISED K -MEANS CLUSTERING USING MIN-MAX
INITIALIZATION
Here it is assumed that the parameters aand bare completely
unknown, except for the sign of a,a>0. Due to the
large uncertainty, we cannot adopt the elementary choice
of the initial values of the centroids µ(1)
ias described in
Section IV. We propose, following the min-max detector
technique described in Subsection III-B, the choice of the
initial centroids µ(1)
i’s using the minimum, miniri, and max-
imum value, maxiri, of the received symbols. The Pearson
code guarantees at least one ‘0’ symbol and also at least one
q1’ symbol in a codeword. The detector may therefore use
the minimum and maximum value of the received symbols as
anchor points defining the range of values of the symbols in
the received vector. To that end, let
α0=min
iriand α1=max
iri.(33)
The qinitial centroids, µ(1)
i, are found by the interpolation
µ(1)
i=α0+(α1α0)i
q1,0iq1.(34)
19324 VOLUME 8, 2020
K. A. S. Immink, K. Cai: Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise
Note that the above initialization step of the modified
k-means clustering technique has the same effect as the scal-
ing used in the min-max detector (11). Figure 4shows results
of computer simulations for the case q=4 and n=64 and a
gain a=1.5. For normalization purposes, we define the SNR
by SNR = −20 log(σ/a). We compared prior art min-max
detector with the k-means clustering detection algorithm. The
detector based on k-means clustering outperforms the prior
art min-max detector.
FIGURE 4. WER for the case q=4, n=64 and gain a=1.5 of a) prior art
min-max detector as described in Subsection III-B, b) k-means clustering
algorithm, and c) upperbound (29) of an ideal fixed threshold detector.
Note that the signal-to-noise ratio is defined by SNR = −20 log(σ/a). The
error performance is independent of the offset b.
In the next subsection, we discuss a second modification
to the basic k-means clustering method using regression
analysis.
C. REVISED K -MEANS CLUSTERING ALGORITHM USING
REGRESSION ANALYSIS
We adopt a second modification to the clustering algorithm
of Section IV. In the basic updating step (23), the kcentroids
are updated by computing a new mean of the members in that
cluster only. Here we assume that the linear channel model,
ri=axi+b+νi, described by (5) holds. We have investi-
gated an alternative method for updating the centroids, µ(t+1)
j,
by applying the well-known linear regression model [23]
that estimates the two coefficients aand binstead of the q
centroids µi.
We start and initialize as described in the previous sub-
section, where the qinitial centroids, µ(1)
i, are found by the
interpolation
µ(1)
i=α0+(α1α0)i
q1,0iq1,(35)
where, as in (33),
α0=min
iriand α1=max
iri.(36)
For the offset only case, a=1, we have
µ(1)
i=α0+i,i=0,...,q1.(37)
After the initialization, we iterate the next two steps until
equilibrium is reached.
Assignment step: Assign the nreceived symbols, ri,
to the ksets V(t+1)
j. If ri, 1 in, is closest to µ(t)
`, or
`=argmin
jriµ(t)
j2
,(38)
then riis assigned to V(t+1)
`. The (temporary) decoded
codeword, denoted by
ˆ
x(t)=(ˆx(t)
1,...,ˆx(t)
n),(39)
is found by
ˆx(t)
i=φV(t)(ri),1in,(40)
where φV(t)(ri)=jsuch that riV(t)
j.
Updating step: Updates of the means µ(t+1)
j,jQ
are found by a linear regression model that estimates
the coefficients aand b. To that end, define the linear
regression model
ˆri= ˆa(t)ˆx(t)
i+ˆ
b(t),(41)
where the (real-valued) regression coefficients ˆa(t)and
ˆ
b(t), chosen to minimize Pn
i=1(ri− ˆri)2,denote the esti-
mates of the unknown quantities aand b. The regres-
sion coefficients ˆa(t)and ˆ
b(t)are found by invoking the
well-known linear regression method [23], and we find
using (13) and (15)
ˆa(t)=Pn
i=1(rir)(ˆxi− ˆx)
Pn
i=1(ˆxi− ˆx)2=σr
σˆx(t)
ρr,ˆ
x(t)(42)
and
ˆ
b(t)=r− ˆa(t)ˆx(t).(43)
We note that for all xS,σˆx(t)6= 0 since Property B
holds, see Subsection VI-A. The updated µ(t+1)
i,i=
0,...,q1, are found by the interpolation
µ(t+1)
i= ˆa(t)i+ˆ
b(t).(44)
For the offset-only case, a=1, we simply find
ˆ
b(t)=r− ˆx(t),(45)
and
µ(t+1)
i=i+ˆ
b(t)=i+r− ˆx(t).(46)
We have conducted a myriad of computer simulations with
the above algorithms. Figure 5compares the error perfor-
mance of the revised k-means clustering using min-max
initialization versus the revised k-means clustering using
regression analysis for the case q=16 and n=64. The
performance difference between the two cluster methods is
independent of the unknown quantities aand b.
VOLUME 8, 2020 19325
K. A. S. Immink, K. Cai: Dynamic Threshold Detection Using Clustering in the Presence of Channel Mismatch and Additive Noise
FIGURE 5. WER of a) revised k-means clustering using min-max
initialization, and b) revised k-means clustering algorithm using
regression method both for q=16 and n=64.
VII. CONCLUSION
We have analyzed k-means clustering technique as a dynamic
threshold detection technique of encoded strings of q-ary
symbols in the presence of signal dependent offset and
additive noise. We have evaluated the error performance
of k-means clustering detection technique, where the signal
dependent offsets are correlated, and can be modelled as
unknown scale, or gain, and translation, or offset. The pro-
posed k-means clustering classification outperforms common
prior art detection methods at the cost of only a slight increase
in (time) complexity.
REFERENCES
[1] S. S. Garani, L. Dolecek, J. Barry, F. Sala, and B. Vasic, ‘‘Signal processing
and coding techniques for 2-D magnetic recording: An overview,’’ Proc.
IEEE, vol. 106, no. 2, pp. 286–318, Feb. 2018.
[2] L. Dolecek and Y. Cassuto, ‘‘Channel coding for nonvolatile memory
technologies: Theoretical advances and practical considerations,’Proc.
IEEE, vol. 105, no. 9, pp. 1705–1724, Sep. 2017.
[3] K. Immink, ‘‘A survey of codes for optical disk recording,’IEEE J. Sel.
Areas Commun., vol. 19, no. 4, pp. 756–764, Apr. 2001.
[4] G. M. Church, Y. Gao, and S. Kosuri, ‘‘Next-generation digital information
storage in DNA,’’ Science, vol. 337, no. 6102, p. 1628, Sep. 2012.
[5] Z. Mei, K. Cai, and X. Zhong, ‘‘Neural network-based dynamic threshold
detection for non-volatile memories,’’ in Proc. IEEE Int. Conf. Commun.
(ICC), Shanghai, China, May 2019, pp. 1–6.
[6] K. A. S. Immink and J. H. Weber, ‘‘Minimum pearson distance detection
for multilevel channels with gain and/or offset mismatch,’’ IEEE Trans. Inf.
Theory, vol. 60, no. 10, pp. 5966–5974, Oct. 2014.
[7] H. Zhou, A. Jiang, and J. Bruck, ‘‘Balanced modulation for non-
volatile memories,’’ Sep. 2012, arXiv:1209.0744. [Online]. Available:
https://arxiv.org/abs/1209.0744
[8] B. Peleato, R. Agarwal, J. M. Cioffi, M. Qin, and P. H. Siegel, ‘‘Adaptive
read thresholds for NAND flash,’’ IEEE Trans. Commun., vol. 63, no. 9,
pp. 3069–3081, Sep. 2015.
[9] C. Cao and I. Fair, ‘‘Mitigation of inter-cell interference in flash memory
with capacity-approaching variable-length constrained sequence codes,’
IEEE J. Sel. Areas Commun., vol. 34, no. 9, pp. 2366–2377, Sep. 2016.
[10] F. Sala, R. Gabrys, and L. Dolecek, ‘‘Dynamic threshold schemes for
multi-level non-volatile memories,’’ IEEE Trans. Commun., vol. 61, no. 7,
pp. 2624–2634, Jul. 2013.
[11] K. A. Schouhamer Immink and K. Cai, ‘‘Composition check codes,’IEEE
Trans. Inf. Theory, vol. 64, no. 1, pp. 249–256, Jan. 2018.
[12] D. Slepian, ‘‘Permutation modulation,’Proc. IEEE, vol. 53, no. 3,
pp. 228–236, Mar. 1965.
[13] K. A. Schouhamer Immink, K. Cai, and J. H. Weber, ‘‘Dynamic threshold
detection based on pearson distance detection,’IEEE Trans. Commun.,
vol. 66, no. 7, pp. 2958–2965, Jul. 2018.
[14] T. Zhao, A. Nehorai, and B. Porat, ‘‘K-means clustering-based data detec-
tion and symbol-timing recovery for burst-mode optical receiver,’’ IEEE
Trans. Commun., vol. 54, no. 8, pp. 1492–1501, Aug. 2006.
[15] A. K. Jain and R. C. Dubes, Algorithms for Clustering Data.
Englewood Cliffs, NJ, USA: Prentice-Hall, 1998.
[16] I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge,
MA, USA: MIT Press, 2016.
[17] F. M. Puglisi, N. Zagni, L. Larcher, and P. Pavan, ‘‘Random telegraph
noise in resistive random access memories: Compact modeling and
advanced circuit design,’IEEE Trans. Electron Devices, vol. 65, no. 7,
pp. 2964–2972, Jul. 2018.
[18] G. Dong, N. Xie, and T. Zhang, ‘‘Enabling NAND flash memory use
soft-decision error correction codes at minimal read latency overhead,’’
IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 60, no. 9, pp. 2412–2421,
Sep. 2013.
[19] K. Immink, ‘‘Coding schemes for multi-level Flash memories that are
intrinsically resistant against unknown gain and/or offset using reference
symbols,’Electron. Lett., vol. 50, no. 1, pp. 20–22, Jan. 2014.
[20] E. W. Forgy, ‘‘Cluster analysis of multivariate data: Efficiency versus
interpretability of classifications,’Biometrics, vol. 21, no. 3, pp. 768–769,
1965.
[21] L. Bottou and Y. Bengio, ‘‘Convergence properties of the k-mean algo-
rithms,’’ in Proc. Adv. Neural Inf. Process. Syst. Cambridge, MA, USA:
MIT Press, 1995, pp. 585–592.
[22] C. Cao and I. Fair, ‘‘Capacity-approaching variable-length pearson codes,’’
IEEE Commun. Lett., vol. 22, no. 7, pp. 1310–1313, Jul. 2018.
[23] R. V. Hogg and A. T. Craig, Introduction to Mathematical Statistics, 5th ed.
New York, NY, USA: Macmillan, 1995.
KEES A. SCHOUHAMER IMMINK (Fellow,
IEEE) received the M.Eng. and Ph.D. degrees
from the Eindhoven University of Technology,
and the Ph.D. degree (Hons.) from the University
of Johannesburg, in 2014. In 1998, he founded
Turing Machines Inc., an innovative start-up
focused on novel signal processing for DNA-based
storage, where he is currently the President.
He received the Knighthood, in 2000, the Personal
Emmy Award, in 2004, the 2017 IEEE Medal of
Honor, the 1999 AES Gold Medal, the 2004 SMPTE Progress Medal,
the 2014 Eduard Rhein Prize for Technology, and the 2015 IET Faraday
Medal. He was inducted into the Consumer Electronics Hall of Fame,
elected into the Royal Netherlands Academy of Sciences, and the (U.S.)
National Academy of Engineering. He served as the President of the Audio
Engineering Society, New York, in 2003.
KUI CAI (Senior Member, IEEE) received the
B.E. degree in information and control engineering
from Shanghai Jiao Tong University, Shanghai,
China, and the joint Ph.D. degree in electri-
cal engineering from the Technical University of
Eindhoven, The Netherlands, and the National
University of Singapore. She is currently an Asso-
ciate Professor with the Singapore University
of Technology and Design (SUTD). Her main
research interests are in the areas of coding theory,
information theory, and signal processing for various data storage systems
and digital communications. She received the 2008 IEEE Communications
Society Best Paper Award in Coding and Signal Processing for Data Storage.
She served as the Vice-Chair (Academia) of the IEEE Communications
Society, Data Storage Technical Committee (DSTC), from 2015 to 2016.
19326 VOLUME 8, 2020
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Sequences encoded with Pearson codes are immune to channel gain and offset mismatch that cause performance loss in communication systems. In this paper, we introduce an efficient method of constructing capacity-approaching variable-length Pearson codes. We introduce a finite state machine (FSM) description of Pearson codes, and present a variable-length code construction process based on this FSM. We then analyze the code rate, redundancy and the convergence property of our codes. We show that our proposed codes have less redundancy than codes recently described in the literature and that they can be implemented in a straightforward fashion.
Article
Full-text available
We consider the transmission and storage of encoded strings of symbols over a noisy channel, where dynamic threshold detection is proposed for achieving resilience against unknown scaling and offset of the received signal. We derive simple rules for dynamically estimating the unknown scale (gain) and offset. The estimates of the actual gain and offset so obtained are used to adjust the threshold levels or to re-scale the received signal within its regular range. Then, the re-scaled signal, brought into its standard range, can be forwarded to the final detection/decoding system, where optimum use can be made of the distance properties of the code by applying, for example, the Chase algorithm. A worked example of a spin-torque transfer magnetic random access memory (STT-MRAM) with an application to an extended (72, 64) Hamming code is described, where the retrieved signal is perturbed by additive Gaussian noise and unknown gain or offset.
Article
Full-text available
A primary source of increased read time on nand flash comes from the fact that, in the presence of noise, the flash medium must be read several times using different read threshold voltages for the decoder to succeed. This paper proposes an algorithm that uses a limited number of rereads to characterize the noise distribution and recover the stored information. Both hard and soft decoding are considered. For hard decoding, this paper attempts to find a read threshold minimizing bit error rate (BER) and derives an expression for the resulting codeword error rate. For soft decoding, it shows that minimizing BER and minimizing codeword error rate are competing objectives in the presence of a limited number of allowed rereads, and proposes a tradeoff between the two. The proposed method does not require any prior knowledge about the noise distribution but can take advantage of such information when it is available. Each read threshold is chosen based on the results of previous reads, following an optimal policy derived through a dynamic programming backward recursion. The method and results are studied from the perspective of an SLC Flash memory with Gaussian noise, but this paper explains how the method could be extended to other scenarios.
Article
Full-text available
The performance of certain transmission and storage channels, such as optical data storage and nonvolatile memory (flash), is seriously hampered by the phenomena of unknown offset (drift) or gain. We will show that minimum Pearson distance (MPD) detection, unlike conventional minimum Euclidean distance detection, is immune to offset and/or gain mismatch. MPD detection is used in conjunction with (T) -constrained codes that consist of (q) -ary codewords, where in each codeword (T) reference symbols appear at least once. We will analyze the redundancy of the new (q) -ary coding technique and compute the error performance of MPD detection in the presence of additive noise. Implementation issues of MPD detection will be discussed, and results of simulations will be given.
Article
In this paper, we report about the derivation of a physics-based compact model of random telegraph noise (RTN) in HfO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -based resistive random access memory (RRAM) devices. Starting from the physics of charge transport, which is different in the high resistive states and low resistive states, we explore the mechanisms responsible for RTN exploiting a hybrid approach, based on self-consistent physics simulations and geometrical simplifications. Then, we develop a simple yet effective physics-based compact model of RTN valid in both states, which can be steadily integrated in state-of-the-art RRAM compact models. The RTN compact model predictions are validated by comparison with both a large experimental data set obtained by measuring RRAM devices in different conditions, and data reported in the literature. In addition, we show how the model enables advanced circuit simulations by exploring three different circuits for memory, security, and logic applications.
Article
Two-dimensional magnetic recording (TDMR) is an emerging storage technology that aims to achieve areal densities on the order of 10 Tb/in <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , mainly driven by innovative channels engineering with minimal changes to existing head/media designs within a systems framework. Significant additive areal density gains can be achieved by using TDMR over bit patterned media (BPM) and energy-assisted magnetic recording (EAMR). In TDMR, the sectors are inherently 2-D with reduced track pitch and bit widths, leading to severe 2-D intersymbol interference (ISI). This necessitates the development of powerful 2-D signal processing and coding algorithms for mitigating 2-D ISI, timing artifacts, jitter, and electronics noise resulting from irregular media grain positions and read-head electronics. The algorithms have to be eventually realized within a read/write channel architecture as a part of a system-on-chip (SoC) within the disk controller system. In this work, we provide a wide overview of TDMR technology, channel models and capacity, signal processing algorithms (detection and timing recovery), and error-correcting codes attuned to 2-D channels. The innovations and advances described not only make TDMR a promising future technology, but may serve a broader engineering audience as well.
Article
Every bit of information in a storage or memory device is bound by a multitude of performance specifications, and is subject to a variety of reliability impediments. At the other end, the physical processes tamed to remember our bits offer a constant source of risk to their reliability. These include a variety of noise sources, access restrictions, inter-cell interferences, cell variabilities, and many more issues. Tying together this vector of performance figures with that vector of reliability issues is a rich matrix of emerging coding tools and techniques. Channel coding schemes ensure target reliability and performance and have been at the core of memory systems since their nascent age. In this survey, we first overview the fundamentals of channel coding and summarize well-known codes that have been used in non-volatile memories (NVMs). Next, we demonstrate why the conventional coding approaches ubiquitously based on symmetric channel models and optimization for the Hamming metric fail to address the needs of modern memories. We then discuss several recently proposed innovative coding schemes. Behind each coding scheme lies an interesting theoretical framework, building on deep ideas from mathematics and the information sciences. We also survey some of the most fascinating bridges between deep theory and storage performance. While the focus of this survey is primarily on the pervasive multi-level NAND Flash, we envision that other benefiting memory technologies will include phase change memory, resistive memories, and others.
Article
We propose using variable-length constrained sequence codes to mitigate inter-cell interference (ICI) in all-bitline flash memory with multi-page programming for single-level cell (SLC), multi-level cell (MLC), and triple-level cell (TLC) flash memory structures. We outline constraints that mitigate ICI in these systems based on an observation of the Gray mapping of data symbols, and we derive the capacity of each constraint. Based on a finite state machine representation of each constraint, we construct variable-length constrained sequence codes with code rates very close to capacity to mitigate ICI in these flash memories. We then exploit the inherent error control capability of the proposed constrained sequence codes to alleviate error propagation. Finally we integrate these codes with error control codes and present simulation results that demonstrate the enhanced bit error rate performance that can be achieved.