PreprintPDF Available

Fragile Complexity of Adaptive Algorithms

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract

The fragile complexity of a comparison-based algorithm is $f(n)$ if each input element participates in $O(f(n))$ comparisons. In this paper, we explore the fragile complexity of algorithms adaptive to various restrictions on the input, i.e., algorithms with a fragile complexity parameterized by a quantity other than the input size n. We show that searching for the predecessor in a sorted array has fragile complexity ${\Theta}(\log k)$, where $k$ is the rank of the query element, both in a randomized and a deterministic setting. For predecessor searches, we also show how to optimally reduce the amortized fragile complexity of the elements in the array. We also prove the following results: Selecting the $k$-th smallest element has expected fragile complexity $O(\log \log k)$ for the element selected. Deterministically finding the minimum element has fragile complexity ${\Theta}(\log(Inv))$ and ${\Theta}(\log(Runs))$, where $Inv$ is the number of inversions in a sequence and $Runs$ is the number of increasing runs in a sequence. Deterministically finding the median has fragile complexity $O(\log(Runs) + \log \log n)$ and ${\Theta}(\log(Inv))$. Deterministic sorting has fragile complexity ${\Theta}(\log(Inv))$ but it has fragile complexity ${\Theta}(\log n)$ regardless of the number of runs.
arXiv:2102.00338v1 [cs.DS] 30 Jan 2021
Fragile Complexity of Adaptive Algorithms
Prosenjit Bose1[0000000218250097] , Pilar Cano2 [0000000243185282], Rolf
Fagerberg3[0000000310043314] , John Iacono2,4[0000000188858172], Riko
Jacob5[0000000194701809] , and Stefan Langerman2[0000000169993088]
1School of Computer Science, Carleton University, Canada. jit@scs.carleton.ca
2Université libre de Bruxelles, Belgium. {pilar.cano, jiacono,
stefan.langerman}@ulb.ac.be
3University of Southern Denmark, Denmark. rolf@imada.sdu.dk
4New York University, USA
5IT University of Copenhagen, Denmark. rikj@itu.dk
Abstract. The fragile complexity of a comparison-based algorithm is
f(n)if each input element participates in O(f(n)) comparisons. In this
paper, we explore the fragile complexity of algorithms adaptive to var-
ious restrictions on the input, i.e., algorithms with a fragile complexity
parameterized by a quantity other than the input size n. We show that
searching for the predecessor in a sorted array has fragile complexity
Θ(log k), where kis the rank of the query element, both in a random-
ized and a deterministic setting. For predecessor searches, we also show
how to optimally reduce the amortized fragile complexity of the ele-
ments in the array. We also prove the following results: Selecting the kth
smallest element has expected fragile complexity O(log log k)for the ele-
ment selected. Deterministically finding the minimum element has fragile
complexity Θ(log(Inv)) and Θ(log(Runs)), where Inv is the number of
inversions in a sequence and Runs is the number of increasing runs in
a sequence. Deterministically finding the median has fragile complex-
ity O(log(Runs) + log log n)and Θ(log(Inv)). Deterministic sorting has
fragile complexity Θ(log(Inv)) but it has fragile complexity Θ(log n)re-
gardless of the number of runs.
Keywords: Algorithms · Comparison based algorithms · Fragile com-
plexity.
1 Introduction
Comparison-based algorithms have been thoroughly studied in computer science.
This includes algorithms for problems such as Minimum,Median,Sorting,
Searching,Dictionaries,Priority Queues, and many others. The cost
measure analyzed is almost always the total number of comparisons performed
by the algorithm, either in the worst case or the expected case. Recently, another
type of cost measure has been introduced [1] which instead considers how many
comparisons each individual element is subjected during the course of the algo-
rithm. In [1], a comparison-based algorithm is defined to have fragile complexity
2 P. Bose, P. Cano, R. Fagerberg, J. Iacono, R. Jacob, and S. Langerman
f(n)if each individual input element participates in at most f(n)comparisons.
The fragile complexity of a computational problem is the best possible fragile
complexity of any comparison-based algorithm solving the problem.
This cost measure has both theoretical and practical motivations. On the
theoretical side, it raises the question of to what extent the comparisons neces-
sary to solve a given problem can be spread evenly across the input elements.
On the practical side, this question is relevant in any real world situation where
comparisons involve some amount of destructive impact on the elements being
compared (hence the name of the cost measure). As argued in [1], one example of
such a situation is ranking of any type of consumable objects (wine, beer, food,
produce), where each comparison reduces the available amount of the objects
compared. Here, an algorithm like QuickSort, which takes a single object and
partitions the whole set with it, may use up this pivot element long before the
algorithm completes. Another example is sports, where each comparison consti-
tutes a match and takes a physical toll on the athletes involved. If a comparison
scheme subjects one contestant to many more matches than others, both fairness
to contestants and quality of result are impacted—finding a winner may not be
very useful if this winner has a high risk of being injured in the process. The
negative impact of comparisons may also be of non-physical nature, for instance
when there is a privacy risk for the elements compared, or when bias grows if
few elements are used extensively in comparisons.
1.1 Previous work
In [1], the study of algorithms’ fragile complexity was initiated and a number of
upper and lower bounds on the fragile complexity for fundamental problems was
given. The problems studied included Minimum, the Selection,Sorting, and
Heap Construction, and both deterministic and randomized settings were
considered. In the deterministic setting, Minimum was shown to have fragile
complexity (log n)and Sorting to have fragile complexity O(log n). Since
Sorting can solve Selection, which can solve Minimum, the fragile complex-
ity of all three problems is Θ(log n). The authors then consider randomized
algorithms, as well as a more fine-grained notion of fragile complexity, where
the objective is to protect selected elements such as the minimum or median
(i.e., the element to be returned by the algorithm), possibly at the expense of
the remaining elements. Among other results, it is shown in [1] that Minimum
can be solved incurring expected O(1) comparisons on the minimum element
itself, at a price of incurring expected O(nε)on each of the rest. Also a more
general trade-off between the two costs is shown, as well as a close to matching
lower bound. For Selection, similar results are given, including an algorithm
incurring expected O(log log n)comparisons on the returned element itself, at a
price of incurring expected O(n)on each of the rest.
An earlier body of work relevant for the concept of fragile complexity is the
study of sorting networks, started in 1968 by Batcher [5]. In sorting networks,
and more generally comparator networks, the notion of depth (the number of
layers, where each layer consists of non-overlapping comparators) and size (the
Fragile Complexity of Adaptive Algorithms 3
total number of comparators) correspond to fragile complexity and standard
worst case complexity, respectively, in the sense that a network with depth f(n)
and size s(n)can be converted into a comparison-based algorithm with fragile
complexity f(n)and standard complexity s(n)by simply simulating the network.
Batcher, as well as a number of later authors [9,16,17,20], gave sorting net-
works with O(log2n)depth and O(nlog2n)size. For a long time it was an
open question whether better results were possible. In 1983, Ajtai, Komlós,
and Szemerédi [2,3] answered this in the affirmative by constructing a sort-
ing network of O(log n)depth and O(nlog n)size. This construction is quite
complex and involves expander graphs [22,23]. It was later modified by oth-
ers [8,12,18,21], but finding a simple, optimal sorting network, in particular one
not based on expander graphs, remains an open problem. Comparator networks
for other problems, such as selection and heap construction have also been stud-
ied [4,6,15,19,26].
While comparator networks are related to fragile complexity in the sense
that results for comparator networks can be transfered to the fragile complexity
setting by simple simulation, it is demonstrated in [1] that the two models are not
equivalent: there are problems where one can construct fragile algorithms with
the same fragile complexity, but with strictly lower standard complexity (i.e.,
total number of comparisons) than what is possible by simulation of comparison
networks. These problems include Selection and Heap Construction.
1.2 Our Contribution
In many settings, the classical worst case complexity of comparison-based al-
gorithms can be lowered if additional information on the input is known. For
instance, sorting becomes easier than Θ(nlog n)if the input is known to be close
to sorted. Another example is searching in a sorted set of elements, which be-
comes easier than O(log n)if we know an element of rank close to the element
searched for. Such algorithms may be described as adaptive to input restrictions
(using the terminology from the sorting setting [10]). Given that the total num-
ber of comparisons can be lowered in such situations, the question arises whether
also reductions in the fragile complexity are possible under these types of input
restrictions.
In this paper, we expand the study of the fragile complexity of comparison-
based algorithms to consider the impact of a number of classic input restrictions.
We show that searching for the predecessor in a sorted array has fragile com-
plexity Θ(log k), where kis the rank of the query element, both in a randomized
and a deterministic setting. For predecessor searches, we also show how to opti-
mally reduce the amortized fragile complexity of the elements in the array. We
also prove the following results: Selecting the kth smallest element has expected
fragile complexity O(log log k)for the element selected. Deterministically find-
ing the minimum element has fragile complexity Θ(log(Inv)) and Θ(log(Runs)),
where Inv is the number of inversions in a sequence and Runs is the number of
increasing runs in a sequence. Deterministically finding the median has fragile
complexity O(log(Runs) + log log n)and Θ(log(Inv)). Deterministic sorting has
4 P. Bose, P. Cano, R. Fagerberg, J. Iacono, R. Jacob, and S. Langerman
fragile complexity Θ(log(Inv)) but it has fragile complexity Θ(log n)regardless
of the number of runs.
2 Searching
The problem of predecessor searching is, given a sorted array Awith nelements,
A[0]..A[n1], answer queries of the form “What is the index of the largest element
in Asmaller than x?” Binary search is the classic solution to the predecessor
search problem. It achieves log nfragile complexity for x, and fragile complexity
at most one for each element of A. We can improve on this in two ways. The
first is where we try to keep the fragile complexity of xsmall, which is possible if
we know something about the rank of x. We show that the optimal dependency
on the rank of xis Θ(log k)where kis its rank, both for deterministic and
randomized algorithms.6The second setting is where we are concerned with the
fragile complexity of the other elements. While there is no way to improve a single
search, classical deterministic binary search will always do the first comparison
with the same element (typically the median). Hence we consider deterministic
algorithms that improve the amortized fragile complexity of any element of the
array Aover a sequence of searches.
2.1 Single search
Theorem 1. Let Abe a sorted array. Determining the predecessor of an element
xwithin Ahas fragile complexity Θ(log k)for deterministic and randomized
algorithms, where kis the rank of xin A.
Proof. The upper bound follows from standard exponential search [11]: We com-
pare xto A[2], A[4], A[8],... until we find the smallest isuch that x < A[2i]. We
perform a binary search with the initial interval [2i1,2i]. If xhas the predecessor
A[k], this requires O(log k)comparisons.
For the lower bound assume we have a deterministic algorithm to deter-
mine the rank of an element x. If the answer of the algorithm is k, let Bkbe
the bit-string resulting from concatenating the sequence of the outcomes of the
comparisons performed by the algorithm, the i-th bit Bk[i] = 0 for x < A[k],
otherwise it is 1. Because the algorithm is deterministic and correct, all these
bit-strings are different and they are a code for the numbers 1,...,n. Now, for
any k, consider the uniform distribution on the numbers 0,...,k1, a distribu-
tion with entropy log k. By Shannon’s source coding theorem, the average code
length must be at least log k, i.e., Pk1
i=0 |Bi| ≥ klog k.
For a contradiction, assume there would be an algorithm with |Bi| ≤ log i
(the binary logarithm itself). Then for k > 1,Pk1
i=0 |Bi|< k log k, in contrast
to Shannon’s theorem.
6For simplicity of exposition, we assume the rank is close to one, but the result clearly
holds for rank distance to other positions in A.
Fragile Complexity of Adaptive Algorithms 5
The bound Pk1
i=0 |Bi| ≥ klog kalso holds for randomized algorithms if the
queries are drawn uniformly from [1,...,k], following Yao’s principle: Any ran-
domized algorithm can be understood as a collection of deterministic algorithms
from which the ’real’ algorithm is drawn according to some distribution. Now
each deterministic algorithm has the lower bound, and the average number of
comparisons of the randomized algorithm is a weighted average of these. Hence
the lower bound also holds for randomized algorithms.
2.2 Sequence of searches
As mentioned, in binary search, the median element of the array will be compared
with every query element. Our goal here is to develop a search strategy so as to
ensure that data far away from the query will only infrequently be involved in a
comparison. Data close to the query must be queried more frequently. While we
prove this formally in Theorem 3, it is easy to see that predecessor and successor
of a query must be involved in comparisons with the query in order to answer
the query correctly.
Theorem 2. There is a search algorithm that for any sequence of predecessor
searches x1, x2, . . . , xmin a sorted array Aof size nthe number of comparisons
with any yAis Olog n+Pm
i=1 1
d(xi,y)where d(x, y)is the number of ele-
ments between xand yin A, inclusive. The runtime is O(log n)per search and
the structure uses O(n)bits of additional space.
Proof. We use the word interval to refer to a contiguous range of A; when we
index an interval, we are indexing Arelative to the start of the interval. Call an
aligned interval Iof Aof rank ito be (A[k·2i]. . . A[(k+ 1) ·2i]) for some integer
k, i.e., the aligned intervals of Aare the dyadic intervals of A. There are O(n)
aligned intervals of A, and for each aligned interval Iof rank iwe store an offset
I.offset which is in the range [0,2i), and it is initialized to 0.
The predecessor search algorithm with query xis a variant of recursive binary
search, where at each step an interval Iqof Ais under consideration, and the
initial recursive call considers the whole array A. Each recursive call proceeds as
follows: Find the largest isuch that there are at least three rank-ialigned inter-
vals in Iq, use Imto denote the middle such interval (or an arbitrary non-extreme
one if there are more than three), and we henceforth refer to this recursive call as
a rank-irecursion. Compare Im[Im.offset]with x, and then increment Im.offset
modulo 2i. Based on the result of the comparison, proceed recursively as in bi-
nary search. The intuition is by moving the offset with every comparison, this
prevents a single element far from the search from being accessed too frequently.
We note that the total space used by the offsets is O(n)words, which can be
reduced to O(n)bits if the offsets are stored in a compact representation.
First, several observations:
1. In a rank-irecursion, Iqhas size at least 3·2i(since there must be at least
three rank-i, size 2ialigned intervals in Iq) and at most 8·2i, the latter
being true as if it was this size there would be three rank-i+ 1 intervals in
Iq, which would contradict Imhaving rank i.
6 P. Bose, P. Cano, R. Fagerberg, J. Iacono, R. Jacob, and S. Langerman
2. If Iqhas size kthen if there is a recursive call, it is called with an interval
of size at most 7
8k. This is true by virtue of Imbeing rank-ialigned with at
least one rank-ialigned interval on either side of Imin i. Since Iqhas size at
most 8·2i, this guarantees an eighth of the elements of Iqwill be removed
from consideration as a result of the comparison in any recursive call.
3. From the previous two points, one can conclude that for a given rank i,
during any search there are at most 7 recursions with of rank i. This is
because after eight recursions any rank-isearch will be reduced below the
minimum for rank i:8·2i·7
88<3·2i.
For the analysis, we fix an arbitrary element yin Aand use the potential
method to analyse the comparisons involving y. Let Iy={I1
y, I2
y...}be the
O(log n)aligned intervals that contain y, numbered such that Ii
yhas rank i.
Element ywill be assigned a potential relative to each aligned interval Ii
y∈ Iy
which we will denote as ϕy(Ii
y). Let ty(Ii
y)be number of times Ii
y.offset needs
to be incremented before Ii
y[Ii
y.offset] = y, which is in the range [0,2i). The
potential relative to Ii
yis then defined as ϕy(Ii
y):=2ity(Ii
y)
2i, and the potential
relative to yis defined to be the sum of the potentials relative to the intervals
in Iy:ϕy:=PIi
y∈Iyϕy(Ii
y).
How does ϕy(Ii
y)change during a search? First, if there is no rank-irecursive
call during the search to an interval containing y, it does not change as Ii
y.offset
is unchanged. Second, observe from point 3 that a search can increase ϕy(Ii
y)by
only 7
2i. Furthermore if ywas involved in a comparison during a rank-irecursion,
there will be a loss of 11
2iunits of potential in ϕy(Ii
y)as the offset of Ii
ychanges
from 0 to 2i1.
Following standard potential-based amortized analysis, the amortized num-
ber of comparisons involving yduring a search is the actual number of compar-
isons (zero or one) plus the change in the potential ϕy. Let imin be the smallest
value of ifor which there was a rank-irecursion that included y. As the maximum
gain telescopes, the potential gain is at most 14
2imin , minus 1 if ywas involved in
a comparison. Thus the amortized number of comparisons with yin the search
is at most 14
2imin .
Observe that if there was a rank-irecursion that included y, that d(x, y )is
at most 8·2iby point 1. This gives d(x, y)8·2i8·2imin . Thus the amortized
cost can be restated as being at most 14
2imin 112
d(x,y).
To complete the proof, the total number of comparisons involving yover a
sequence of searches is the sum of the amortized costs plus any potential loss. As
the potential ϕyis always nonnegative and at most log n(1 for each ϕy(Ii
y)),
this gives the total cost as Olog n+Pm
i=1 1
d(xi,y).
Note that the above proof was designed for easy presentation and not an
optimal constant. Also note that this theorem implies that if the sequence of
searches is uniformly random, the expected fragility of all elements is O(log n
n),
which is asymptotically the best possible since random searches require (log n)
comparisons in expectation.
Fragile Complexity of Adaptive Algorithms 7
2.3 Lower Bounds.
It is well-known that comparison-based searching requires (log n)comparisons
per search. In our method, taking a single search xisumming over the upper
bound on amortized cost of the number of comparisons with y,42
d(xi,y), for all y
yields a harmonic series which sums to O(log n). But we can prove something
stronger:
Theorem 3. There is a constant csuch that if a predecessor search algorithm
has an amortized number of comparisons of f(d(xi, y)) for an arbitrary yfor
every sequence of predecessor searches x1, x2,...xm, then Pp
k=1 f(k)clog p
for all pn.
Proof. This can be seen by looking at a random sequence of predecessor searches
for which the answers are uniform among A[0] ...A[p1], if the theorem was
false, similarly to the proof of Theorem 1, this would imply the ability to execute
such a sequence in o(log p)amortized time per operation.
This shows that a flatter asymptotic tradeoff between d(xi, y)and the amor-
tized comparison cost is impossible; more comparisons are needed in the vicinity
of the search than farther away. For example, a flat amortized number of compar-
isons of log n
nfor all elements would sum up to O(log n)amortized comparisons
over all elements, but yet would violate this theorem.
2.4 Extensions.
Here we discuss extensions to the search method above. We omit the proofs as
they are simply more tedious variants of the above.
One can save the additional space used by the offsets of the intervals through
the use of randomization. The offsets force each item in the interval to take its
turn as the one to be compared with, instead one can pick an item at random
from the interval. This can be further simplified into a binary search where at
each step one simply picks a random element for the comparison amongst those
(in the middle half) of the part of the array under consideration.
To allow for insertions and deletions, two approaches are possible. The first
is to keep the same array-centric view and simply use the packed-memory array
[14,24,25] to maintain the items in sorted order in the array. This will give rise
to a cost of O(log2n)time which is inherent in maintaining a dynamic collection
of items ordered in an array [7] (but no additional fragility beyond searching
for the item to insert or delete as these are structural changes). The second
approach would be to use a balanced search tree such as a red-black tree [13].
This will reduce the insertion/deletion cost to O(log n)but will cause the search
cost to increase to O(log2n)as it will take O(log n)time to move to the item
in each interval indicated by the offset, or to randomly choose an item in an
interval. The intervals themselves would need to allow insertions and deletions,
and would, in effect be defined by the subtrees of the red-back tree. It remains
open whether there is a dynamic structure with the fragility results of Theorem 2
where insertions and deletions can be done in O(log n)time.
8 P. Bose, P. Cano, R. Fagerberg, J. Iacono, R. Jacob, and S. Langerman
3 Selection
In this section we consider the problem of finding the k-th smallest element of an
unsorted array. There is a randomized algorithm that selects the k-th smallest
element with expected fragile complexity of O(log log n)for the selected ele-
ment [1]. We consider the question if this complexity can be improved for small
k. In this section we define a sampling method that, combined with the algo-
rithm given in [1], selects the k-th smallest element with expected O(log log k)
comparisons.
Next, we define the filtering method ReSet in a tail-recursive fashion. The
idea of this procedure is the following: First, we build a random half size sample
A1from the input set X. Later, we continue recursively constructing a random
half sample Aifrom the previous sample Ai1until we get a random sample
Aof size O(k+ 1). Once Ais given, then a set A
of size O(k)is given for
the previous recursive call. Using such set, a new subset A
1is given from the
previous sample A1where its expected size is O(k). This process continuous
until a final subset Cis given from the input set Xsuch that its expected size
O(k)and it contains the k-th smallest element of X.
1: procedure ReSet(X, k)Returns a small subset Cof Xthat contains the k-th
element
2: Let n=|X|and C=
3: if kn
21The set has size O(k+ 1)
4: Let A=X
5: else Recursively construct a sample of expected size O(k+ 1)
6: Sample Auniformly at random from X,|A|=n
2
7: Let A=ReSet(A, k)
8: Choose the (k+ 1)-th smallest element zfrom A(by standard linear time
selection)
9: Let C={xX:xz}
10: return C
In the following theorem we show that the combination of the ReSet proce-
dure and the randomized selection algorithm in [1], results in expected O(log log k)
comparisons for the selected element.
Theorem 4. Randomized selection is possible in expected fragile complexity
O(log log k)in the selected element.
Proof. Let us show that the following procedure for selecting the k-th element
in a set Xwith |X|=n, gives an expected fragile complexity O(log log k)in the
k-th element:
If k > n 1
100 , then let S=X. If kn1
100 , then sample uniformly at random
Sfrom X, where |S|=n
k. Let C=ReSet(S, k) and select the k+ 1-th smallest
element zfrom Cby standard linear time selection. Let S={xX:xz}.
Finally, apply to Sthe randomized selection algorithm of [1].
Let xkdenote the k-th smallest element in Xand let fkdenote the fragile
complexity of xk. Note that if xkS, then, before constructing S,fkis given
by the fragile complexity of xkin ReSet(S, k)plus O(|C|)when finding the
Fragile Complexity of Adaptive Algorithms 9
(k+ 1)-th smallest element in C. Otherwise, xkis not compared until Sis con-
structed. On the other hand, recall that the expected fkin the algorithm in [1]
is O(log log m)where mis the size of the input set. Hence, the expected fkafter
selecting the k+ 1-th element in Cis 1 when creating Splus the expected fk
in the randomized selection algorithm in [1] that is P|S|O(log log |S|)P[|S|] =
E[O(log log |S|)]. Thus, E[fk] = (E[fkin ReSet|xkS] + E[|C|])P[xkS] +
0P[xk/S] + 1 + E[O(log log |S|)]. Since the logarithm is a concave function,
E[O(log log |S|)] O(log log(E[|S|])). Therefore, if we prove that: (i) the ex-
pected fragile complexity of xkbefore creating Sis O(1) and (ii) E[|S|] = ckc
for some constants cand c. Then, we obtain that E[fk]O(1)+1+O(clog log k+
log c) = O(log log k), as desired. In order to prove (i) and (ii) we consider 2 cases:
(1) k > n 1
100 , (2) kn1
100 .
Case 1) S=Xand it makes no previous comparisons in any element, proving
(i). In addition, Shas size less than k100. Thus, (ii) holds.
Case 2) Sis a sample of Xwith size n
kand S=ReSet(S, k).
First, let us show (i). If xk/S, then there are no previous comparisons.
Hence, the expected fragile complexity of xkbefore constructing Sis given by
(E[fkin ReSet|xkS] + E[|C|])P[xkS] + 0. Since Sis an uniform random
sample with size n
k,P[xkS] = 1
k, it suffices to show that E[fkin ReSet|xk
S] + E[|C|] = O(k), which gives an expectation of O(k)1
k=O(1), proving (i).
So, let us show that E[fkin ReSet|xkS] + E[|C|] = O(k). Let A0=Sand
let A1be the sample of A0when passing through line 6 in ReSet. Similarly,
denote by Aito the sample of Ai1in the i-th recursive call of ReSet and let
A
i=ReSet(Ai, k). Note that by definition A
0=C. Let + 1 be the number of
recursive calls in ReSet(S, k).
Since Aiis a uniform random sample of size |Ai1|
2for all i1,P[x
Ai|xAi1] = 21and P[xAi|x /Ai1] = 0. Hence, P[xkAi] = P[xk
i
i=0Ai] = 2i. Note that the number of comparisons of xkin ReSet is given by
the number of times xkis compared in lines 8 and 9. Thus, for each i-th recursive
call: if xkAi, then xkis compared once in line 9; and if xkAiA
i, then
xkis compared at most |A
i|times in line 8. Otherwise, xkis not compared in
that and the next iterations. Thus, E[fkin ReSet|xkS] + E[|C|]P
i=0(1 +
E[|A
i|])P[xkAi] = P
i=0 2i(1 + E[|A
i|]) 2(1 + E[|A
i|]). Let us compute
E[|A
i|]. Since the (+1)-th iteration ReSet(A, k)passes through the if in line
3, there is no new sample from A. Thus, A
is given by the k+1 smallest elements
of A. Therefore, E[|A
|] = k+1 Denote by ai
jto the j-th smallest element of A
i.
For the case of 0i < ℓ, we have A
i={xAi+1 :xai+1
k+1 }. Hence, E[|A
i|] =
E[|{xAi+1 :xai+1
k+1 }|] = E[|{xAi+1 :xai+1
1]}|] + Pk
j=1 E[|{x
Ai+1 :ai+1
j1< x ai+1
j]}|]Pk+1
j=1 P
t=1 t21(2(t1)) = 2(k+ 1). Therefore,
E[fkin ReSet|xkS] + E[|C|] = P
i=0 2i(1 + E[|A
i|]) 2 + 2E[|A
i|] = O(k)
proving (i). Finally, let us show (ii): For simplicity, let cjdenote the j-th smallest
element of C. Then, E[|S|] = E[|{xX:xc1}|] + Pk
j=1 E[|{xX:cjx
cj+1}|]Pk+1
j=1 P
j=0 jk1(1 k1)j1=k(k+ 1) = O(k2), proving (ii).
10 P. Bose, P. Cano, R. Fagerberg, J. Iacono, R. Jacob, and S. Langerman
4 Sorting
When the input is known to have some amount of existing order, sorting can
be done faster than Θ(nlog n). Quantifying the amount of existing order is tra-
ditionally done using measures of disorder [10], of which Inv and Runs are two
classic examples.7A sorting algorithm is adaptive to a measure of disorder if
it is faster for inputs with a smaller value of the measure. For the above mea-
sures, run times of O(nlog(Inv/n)) and O(nlog(Runs)) can be achieved. These
results are best possible for comparison-based sorting, by standard information-
theoretic arguments based on the number of different inputs having a given
maximal value of the measure.
The fact [3,1] that we can sort all inputs in Θ(nlog n)time and Θ(log n)
fragile complexity can be viewed as being able to distribute the necessary com-
parisons evenly among the elements such that each element takes part in at most
Θ(log n)comparisons. Given the running times for adaptive sorting stated above,
it is natural to ask if for an input with a given value of Inv or Runs we are able
to sort in a way that distributes the necessary comparisons evenly among the
elements, i.e., in a way such that each element takes part in at most O(log(Inv))
or O(log(Runs)) comparisons, respectively. In short, can we sort in fragile com-
plexity O(log(Inv)) and O(log(Runs))? Or more generally, what problems can
we solve with fragile complexity adaptive to Inv and Runs? In this section, we
study the fragile complexity of deterministic algorithms for Minimum,Median,
and Sorting and essentially resolve their adaptivity to Inv and Runs.
Theorem 5. Minimum has fragile complexity Θ(log(Runs)).
Proof. For the upper bound: identify the runs in O(1) fragile complexity by a
scan of the input. Then, use a tournament on the heads of the runs since the
minimum is the minimum of the heads of the runs. For the lower bound: apply
the logarithmic lower bound for Minimum [1] on the heads of the runs.
Theorem 6. Sorting has fragile complexity Θ(log n), no matter what value of
Runs is assumed for the input.
Proof. The upper bound follows from general sorting. For the lower bound: the
input consisting of a run Rof length n1and one more element xhas Runs = 2,
but log ncomparisons on xcan be forced by an adversary before the position of
xin Ris determined.
Theorem 7. Median has fragile complexity O(log(Runs) + log log n).
Proof. Assume that 4·Runs ·log n < n/2, since otherwise the claimed fragile
complexity is O(log n)for which we already have a median algorithm [1]. Con-
sider the rank space [1, n](i.e., the indices of the input elements in the total
7The measure Inv is defined as the total number of inversions in the input, where
each of the n
2pairs of elements constitute an inversion if the elements of the pair
appear in the wrong order. The measure Runs is defined as the number of runs in
the input, where a run is a maximal consecutive ascending subsequence.
Fragile Complexity of Adaptive Algorithms 11
sorted order) of the input elements and consider the rank interval [a, b]around
the median defined by a=n/24·Runs·log nand b=n/2 + 4 ·Runs ·log n. In
each step of the algorithm, elements are removed in two ways: type A removals
and type B removals. A removal of type A is a balanced removal, where a number
of elements with ranks in [1, a 1] are removed and the same number of elements
with ranks in [b+ 1, n]are removed. The key behind the type A removal is that
the median element of the set prior to the removal is the same as the median of
the set after the removal, if the median prior to the removal has a rank in [a, b].
A removal of type B is a removal of elements with arbitrary rank. However,
the total number of elements removed by type B removals is at most 7·Runs·log n
during the entire run of the algorithm. Hence, repeated use of type A and type B
removals will maintain the invariant that the median of the remaining elements
has a rank in [a, b].
We now outline the details of the algorithm. The first step is to identify all
the runs in O(1) fragile complexity by a scan. A run will be considered short if
the run consists of fewer than 7·log nelements and it will be considered long
otherwise. A step of the algorithm proceeds by first performing a type B removal
followed by a type A removal. A type B removal consists of removing all short
runs that are present. The short runs that are removed will be reconsidered again
at the end once the number of elements under consideration by the algorithm is
less than 64 ·Runs ·log n.
Once a type B removal step is completed, only long runs remain under con-
sideration. We now describe a type A removal step. Note that a long run may
become short after a type A removal step, in which case it will be removed
as part of the next type B removal step. Each run can become short (and be
removed by a type B removal) only once, hence the total number of elements
removed by type B removals will be at most 7·Runs ·log n, as claimed.
In the following, let Ndenote the elements under consideration just before a
type A removal (i.e., the elements of the remaining long runs), and let N=|N|.
The algorithm stops when N64 ·Runs ·log n.
To execute the type A removal step, the algorithm divides each long run R
into blocks of length log n. The blocks of a run are partitioned by a partitioning
block. The partitioning block has the property that there are at least |R|/7
elements of Rwhose values are less than the values in the partitioning block and
at least 5|R|/7elements of Rwhose value are greater than the elements in the
partitioning block. One element xRis selected from the partitioning block. We
will refer to this element as a partitioning element. These partitioning elements
are then sorted into increasing order, which incurs a cost of O(log(Runs)) fragile
complexity on each of the partitioning elements. The runs are then arranged in
the same order as their partitioning elements. Label this sequence of runs as
R1, R2,...,Rk, and let tbe the largest index such that Pt1
i=1 |Ri|< N/8.
Since the partitioning element xRtis smaller than all the elements in the
blocks with values greater than their respective partitioning blocks in Rt, Rt+1,
...,Rk, we have that xRtis smaller than (7/8)(5N/7) = 5N/8of the remaining
elements. Hence in rank it is at least N/8below the median of the remaining
12 P. Bose, P. Cano, R. Fagerberg, J. Iacono, R. Jacob, and S. Langerman
elements. By the invariant on the position in rank space of this median and the
fact that N > 64 ·Runs ·log n, we note that xRthas a rank below a. We also
note that all the elements below the partitioning blocks in R1, R2,...,Rthave
value less than xRt. This constitutes at least (1/8)(N/7) = N/56 elements in N
with rank below a. Therefore, we can remove N/56 elements with rank below a.
In a similar manner, we can find at least N/56 elements in Nwith rank above b.
Removal of these 2N/56 = N/28 elements in Nconstitutes a type A removal
step.
Since the number of elements under consideration, i.e. N, decreases by a
constant factor at each step, the algorithm performs O(log n)type A and type B
removal steps before we have N64 ·Runs ·log n. Since each block under
consideration in a type A removal step has size log n, we can guarantee that each
element in a partitioning block only needs to be selected as a partitioning element
O(1) times. This implies that a total cost of O(log(Runs)) fragile complexity is
incurred on each element once we have that N64 ·Runs ·log n.
We now describe the final step of the algorithm. At this point, the algorithm
combines the last Nelements with all the short runs removed during its execu-
tion up to this point, forming the set S. This set is the original elements subjected
to a series of type A removals, each of which are balanced and outside the rank in-
terval [a, b]. Hence, the median of Sis the global median. As |S| =O(Runs·log n),
we can find this median in O(log(Runs·log n)) = O(log(Runs) +log log n)fragile
complexity [1], which dominates the total fragile complexity of the algorithm.
We note that for Runs = 2, we can improve the above result to O(1) fragile
complexity as follows. Let the two runs be R1and R2, with |R1| ≤ |R2|. Compare
their middle elements xand yand assume xy. Then the elements in the
first half of R1are below n/2other elements, and hence are below the median.
Similarly, the elements in the last half of R2are above the median. Hence, we
can remove |R1|/2elements on each side of the median by removing that many
elements from one end of each run. The median of the remaining elements is
equal to the global median. By recursion, we in log |R1|steps end up with R1
reduced to constant length. Then O(1) comparisons with the center area of R2
will find the median. Because both runs lose elements in each recursive step,
both xand ywill be new elements each time. The total fragile complexity of the
algorithm is therefore O(1).
Theorem 8. Minimum has fragile complexity Θ(log(Inv)).
Proof. Lower bound: For any k, consider the instances composed of kelements
in random order followed by nklarger elements in sorted order. These
instances have Inv k. Finding the minimum is equal to finding the minimum
on the first kelements, which has a lower bound [1] of log k=(log k)on
its fragile complexity.
For the upper bound, we will remove a subset Iof size O(Inv) which leaves
a single sorted run R. We can find the mininum in Iin O(log(Inv)) fragile
complexity by a tournament tree, which can then be compared to the head of R
for the final answer.
Fragile Complexity of Adaptive Algorithms 13
We find Iand Rin O(1) fragile complexity during a scan of the input as
follows, using Ras a stack. For each new element escanned, we compare it to the
current top element fof R. If eis larger, we push eto the top of R. If eis smaller,
it forms an inversion with f, and we include ein I. We also put a mark on f. If
an element on the stack gets two marks, we pop it, include it in I, remove one of
its marks (which will account for its inclusion in I) and move the second mark
to the new top element fof R. If fnow has two marks, this process continues
until an element with only a single mark is created (or the stack gets empty).
An element is compared with exactly one element residing earlier in the input
(when the element is scanned). To count comparisons with elements residing
later in the input, call such a comparison large or small, depending on whether
the other element is larger or smaller. It is easy to see that elements on the stack
always have between zero and one marks, that an element with zero marks has
participated in one large comparison, and that an element with one mark has
either participated in at most two larger comparisons or one smaller comparison.
Hence, the fragile complexity of the process is O(1). By the accounting scheme,
Iis no larger than Inv plus the number of marks, which is also at most Inv.
Theorem 9. Median has fragile complexity Θ(log(Inv)).
Proof. As Median solves Minimum via padding with nelements of value −∞,
the lower bound follows from the lower bound on Minimum. For the upper
bound, find Rand Ias in the upper bound for Minimum, sort Iin fragile
complexity O(log(Inv)) and use the algorithm for Median for Runs = 2.
Theorem 10. Sorting has fragile complexity Θ(log(Inv)).
Proof. The lower bound follows from the lower bound on Minimum. For the
upper bound, find Rand Ias in the upper bound for Minimum and let each
element recall its position in the input. Divide the sorted sequence Rinto con-
tiguous blocks of size |I|and let Ribe the set of i’th elements of all blocks.
With the i’th element of Iwe perform an exponential search on Ri, starting
from the block where the element’s position in the input is. If the search moves
a distance k, the element from Iparticipated in at least kinversions in the
input, so kInv and hence the incurred fragile complexity for the element
is O(log k) = O(log(Inv)). A fragile complexity of O(1) is incurred on the ele-
ments of R, as each Riis used once. After this, each element from Iknows its
position in Rwithin a window of size |I|. If the window of an element and a
block overlaps, we call the element and the block associated. Each block of R
is associated with at most |I|=O(Inv) elements, and each element is associ-
ated with at most two blocks. For each block in turn, we now sort its associated
elements and merge them into the block (except for tail elements overlapping
the next block). The sorting incurs O(log(Inv)) fragile complexity, as does the
merging if we use exponential merging [1]. We remove all inserted elements from
any association with the next block. Then we continue with the next block.
14 P. Bose, P. Cano, R. Fagerberg, J. Iacono, R. Jacob, and S. Langerman
Acknowledgements. This material is based upon work performed while attending Al-
goPARC Workshop on Parallel Algorithms and Data Structures at the University of
Hawaii at Manoa, in part supported by the National Science Foundation under Grant
No. CCF-1930579. We thank Timothy Chan and Qizheng He for their ideas improving
the randomized selection algorithm. P.B was partially supported by NSERC. P.C and
J.I. were supported by F.R.S.-FNRS under Grant no MISU F 6001 1. R.F. was par-
tially supported by the Independent Research Fund Denmark, Natural Sciences, grant
DFF-7014-00041. J.I. was supported by NSF grant CCF-1533564. S.L. is Directeur de
Recherches du F.R.S.-FNRS.
References
1. Afshani, P., Fagerberg, R., Hammer, D., Jacob, R., Kostitsyna, I., Meyer, U., Pen-
schuck, M., Sitchinava, N.: Fragile complexity of comparison-based algorithms. In:
Bender, M.A., Svensson, O., Herman, G. (eds.) 27th Annual European Symposium
on Algorithms, ESA 2019, September 9-11, 2019, Munich/Garching, Germany.
LIPIcs, vol. 144, pp. 2:1–2:19. Schloss Dagstuhl - Leibniz-Zentrum für Informatik
(2019)
2. Ajtai, M., Komlós, J., Szemerédi, E.: An O(nlog n)sorting network. In: Proceed-
ings of the 15th Symposium on Theory of Computation. pp. 1–9. STOC ’83, ACM
(1983)
3. Ajtai, M., Komlós, J., Szemerédi, E.: Sorting in clog nparallel steps. Combinatorica
3(1), 1–19 (Mar 1983)
4. Alekseev, V.E.: Sorting algorithms with minimum memory. Kibernetika 5(5), 99–
103 (1969)
5. Batcher, K.E.: Sorting networks and their applications. Proceedings of AFIPS
Spring Joint Computer Conference pp. 307–314 (1968)
6. Brodal, G.S., Pinotti, M.C.: Comparator networks for binary heap construction.
In: Proc. 6th Scandinavian Workshop on Algorithm Theory, LNCS, vol. 1432, pp.
158–168. Springer Verlag, Berlin (1998)
7. Bulánek, J., Koucký, M., Saks, M.E.: Tight lower bounds for the online labeling
problem. SIAM J. Comput. 44(6), 1765–1797 (2015)
8. Chvátal, V.: Lecture notes on the new AKS sorting network. Tech. Rep. DCS-TR-
294, Department of Computer Science, Rutgers University, New Brunswick, NJ
(1992, October)
9. Dowd, M., Perl, Y., Rudolph, L., Saks, M.: The periodic balanced sorting network.
J. ACM 36(4), 738–757 (1989, October)
10. Estivill-Castro, V., Wood, D.: A survey of adaptive sorting algorithms. ACM Com-
put. Surv. 24(4), 441–476 (1992)
11. Fredman, M.L.: Two applications of a probabilistic search technique: Sorting x +
y and building balanced search trees. In: Rounds, W.C., Martin, N., Carlyle, J.W.,
Harrison, M.A. (eds.) Proceedings of the 7th Annual ACM Symposium on Theory
of Computing, May 5-7, 1975, Albuquerque, New Mexico, USA. pp. 240–244. ACM
(1975)
12. Goodrich, M.T.: Zig-zag sort: a simple deterministic data-oblivious sorting algo-
rithm running in O(nlog n)time. In: Shmoys, D.B. (ed.) STOC’14. pp. 684–693.
ACM (2014)
Fragile Complexity of Adaptive Algorithms 15
13. Guibas, L.J., Sedgewick, R.: A dichromatic framework for balanced trees. In: 19th
Annual Symposium on Foundations of Computer Science, Ann Arbor, Michigan,
USA, 16-18 October 1978. pp. 8–21. IEEE Computer Society (1978)
14. Itai, A., Konheim, A.G., Rodeh, M.: A sparse table implementation of priority
queues. In: Even, S., Kariv, O. (eds.) Automata, Languages and Programming,
8th Colloquium, Acre (Akko), Israel, July 13-17, 1981, Proceedings. Lecture Notes
in Computer Science, vol. 115, pp. 417–431. Springer (1981)
15. Jimbo, S., Maruoka, A.: A method of constructing selection networks with O(log n)
depth. SIAM Journal on Computing 25(4), 709–739 (1996)
16. Parberry, I.: The pairwise sorting network. Parallel Processing Letters 2(2-3), 205–
211 (1992)
17. Parker, B., Parberry, I.: Constructing sorting networks from k-sorters. Information
Processing Letters 33(3), 157–162 (30 Nov 1989)
18. Paterson, M.S.: Improved sorting networks with O(log N)depth. Algorithmica
5(1), 75–92 (1990)
19. Pippenger, N.: Selection networks. SIAM Journal on Computing 20(5), 878–887
(1991)
20. Pratt, V.R.: Shellsort and Sorting Networks. Outstanding Dissertations in the
Computer Sciences, Garland Publishing, New York (1972)
21. Seiferas, J.I.: Sorting networks of logarithmic depth, further simplified. Algorith-
mica 53(3), 374–384 (2009)
22. S.Hoory, Linial, N., Wigderson, A.: Expander graphs and their applications.
BAMS: Bulletin of the American Mathematical Society 43, 439–561 (2006)
23. Vadhan, S.P.: Pseudorandomness. Foundations and Trends in Theoretical Com-
puter Science 7(1-3), 1–336 (2012)
24. Willard, D.E.: Good worst-case algorithms for inserting and deleting records in
dense sequential files. In: Zaniolo, C. (ed.) Proceedings of the 1986 ACM SIGMOD
International Conference on Management of Data, Washington, DC, USA, May
28-30, 1986. pp. 251–260. ACM Press (1986)
25. Willard, D.E.: A density control algorithm for doing insertions and deletions in
a sequentially ordered file in good worst-case time. Inf. Comput. 97(2), 150–204
(1992)
26. Yao, A., Yao, F.F.: Lower bounds on merging networks. J. ACM 23(3), 566–571
(1976)
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
We further simplify Paterson’s version of the Ajtai–Komlós–Szemerédi sorting network, and its analysis, mainly by tuning the invariant to be maintained.
Article
Full-text available
We describe and analyze Zig-zag Sort--a deterministic data-oblivious sorting algorithm running in O(n log n) time that is arguably simpler than previously known algorithms with similar properties, which are based on the AKS sorting network. Because it is data-oblivious and deterministic, Zig-zag Sort can be implemented as a simple O(n log n)-size sorting network, thereby providing a solution to an open problem posed by Incerpi and Sedgewick in 1985. In addition, Zig-zag Sort is a variant of Shellsort, and is, in fact, the first deterministic Shellsort variant running in O(n log n) time. The existence of such an algorithm was posed as an open problem by Plaxton et al. in 1992 and also by Sedgewick in 1996. More relevant for today, however, is the fact that the existence of a simple data-oblivious deterministic sorting algorithm running in O(n log n) time simplifies the inner-loop computation in several proposed oblivious-RAM simulation methods (which utilize AKS sorting networks), and this, in turn, implies simplified mechanisms for privacy-preserving data outsourcing in several cloud computing applications. We provide both constructive and non-constructive implementations of Zig-zag Sort, based on the existence of a circuit known as an epsilon-halver, such that the constant factors in our constructive implementations are orders of magnitude smaller than those for constructive variants of the AKS sorting network, which are also based on the use of epsilon-halvers.
Conference Paper
Full-text available
The purpose of this paper is to describe a sorting network of size 0(n log n) and depth 0(log n). A natural way of sorting is through consecutive halvings: determine the upper and lower halves of the set, proceed similarly within the halves, and so on. Unfortunately, while one can halve a set using only 0(n) comparisons, this cannot be done in less than log n (parallel) time, and it is known that a halving network needs (½)n log n comparisons. It is possible, however, to construct a network of 0(n) comparisons which halves in constant time with high accuracy. This procedure (&egr;-halving) and a derived procedure (&egr;-nearsort) are described below, and our sorting network will be centered around these elementary steps.
Conference Paper
Full-text available
To achieve high throughput rates today's computers perform several operations simultaneously. Not only are I/O operations performed concurrently with computing, but also, in multiprocessors, several computing operations are done concurrently. A major problem in the design of such a computing system is the connecting together of the various parts of the system (the I/O devices, memories, processing units, etc.) in such a way that all the required data transfers can be accommodated. One common scheme is a high-speed bus which is time-shared by the various parts; speed of available hardware limits this scheme. Another scheme is a cross-bar switch or matrix; limiting factors here are the amount of hardware (an m × n matrix requires m × n cross-points) and the fan-in and fan-out of the hardware.
Article
The sorting network described by Ajtaiet al. was the first to achieve a depth ofO(logn). The networks introduced here are simplifications and improvements based strongly on their work. While the constants obtained for the depth bound still prevent the construction being of practical value, the structure of the presentation offers a convenient basis for further development.
Article
Ajtai, Koml'os, and Szemer'edi constructed sorting networks with N wires of depthO(log N ). They were not concerned with the value of the proportionality constantimplicit in the O-notation; subsequently Paterson replaced the O(log N) by c log 2 Nwith c under 6100. We describe an implementation of a more recent, and as yetunpublished, proposal of Ajtai, Koml'os, and Szemer'edi, that yields a smaller value ofc: for every integer N such that N 278there is a sorting network on N wires...
Article
Consider a file which arranges records in sequential order, and stores them with possible empty spaces in M consecutive pages of memory. We develop an insertion-deletion algorithm which runs in a worst-case time approximately proportional to log2M divided by the page-size when the set of manipulated records has cardinality O(M).
Article
Comparator networks for constructing binary heaps of size n are presented which have size and depth . A lower bound of for the size of any heap construction network is also proven, implying that the networks presented are within a constant factor of optimal. We give a tight relation between the leading constants in the size of selection networks and in the size of heap construction networks.