Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems

SIAM Journal on Matrix Analysis and Applications (Impact Factor: 1.59). 07/2006; 35(3). DOI: 10.1137/090771430
Source: arXiv

ABSTRACT We present a randomized algorithm that, on input a symmetric, weakly
diagonally dominant n-by-n matrix A with m nonzero entries and an n-vector b,
produces a y such that $\norm{y - \pinv{A} b}_{A} \leq \epsilon \norm{\pinv{A}
b}_{A}$ in expected time $O (m \log^{c}n \log (1/\epsilon)),$ for some constant
c. By applying this algorithm inside the inverse power method, we compute
approximate Fiedler vectors in a similar amount of time. The algorithm applies
subgraph preconditioners in a recursive fashion. These preconditioners improve
upon the subgraph preconditioners first introduced by Vaidya (1990).
For any symmetric, weakly diagonally-dominant matrix A with non-positive
off-diagonal entries and $k \geq 1$, we construct in time $O (m \log^{c} n)$ a
preconditioner B of A with at most $2 (n - 1) + O ((m/k) \log^{39} n)$ nonzero
off-diagonal entries such that the finite generalized condition number
$\kappa_{f} (A,B)$ is at most k, for some other constant c.
In the special case when the nonzero structure of the matrix is planar the
corresponding linear system solver runs in expected time $ O (n \log^{2} n + n
\log n \ \log \log n \ \log (1/\epsilon))$.
We hope that our introduction of algorithms of low asymptotic complexity will
lead to the development of algorithms that are also fast in practice.

51 Reads
  • Source
    • "Using the algorithm of Goldberg and Rao, this results in an algorithm with running time˜O (mn 1/2 ). In [13], Christiano et al. improved this running time tõ O(mn 1/3 ) by applying the almost linear-time Laplacian solver of Spielman and Teng [31] to iteratively minimize a softmax approximation of the edge congestions. Kelner et al. [16] and Sherman [30] independently published two algorithms which allow to compute a (1 + ε)-approximation to an undirected max flow problem in time almost linear in m. "
    [Show abstract] [Hide abstract]
    ABSTRACT: We present a near-optimal distributed algorithm for $(1+o(1))$-approximation of single-commodity maximum flow in undirected weighted networks that runs in $(D+ \sqrt{n})\cdot n^{o(1)}$ communication rounds in the \Congest model. Here, $n$ and $D$ denote the number of nodes and the network diameter, respectively. This is the first improvement over the trivial bound of $O(n^2)$, and it nearly matches the $\tilde{\Omega}(D+ \sqrt{n})$ round complexity lower bound. The development of the algorithm contains two results of independent interest: (i) A $(D+\sqrt{n})\cdot n^{o(1)}$-round distributed construction of a spanning tree of average stretch $n^{o(1)}$. (ii) A $(D+\sqrt{n})\cdot n^{o(1)}$-round distributed construction of an $n^{o(1)}$-congestion approximator consisting of the cuts induced by $O(\log n)$ virtual trees. The distributed representation of the cut approximator allows for evaluation in $(D+\sqrt{n})\cdot n^{o(1)}$ rounds. All our algorithms make use of randomization and succeed with high probability.
  • Source
    • "We use in our analysis the following five basic facts (c.f. [1] [17]). "
    [Show abstract] [Hide abstract]
    ABSTRACT: We propose the first unified framework that yields an improved sparsification algorithm for computing in nearly linear time a spectral sparsifier $D-\widehat{M}_{N}$ for any matrix-polynomial of the form \[ D-\widehat{M}_{N}\approx_{\epsilon}D-D\sum_{i=0}^{N}\gamma_{i}(D^{-1} M)^i \] where $B=D-M\in\mathbb{R}^{n\times n}$ is either Laplacian or SDDM matrix with $m_B$ non-zero entries, and the vector $\gamma$ is induced by a mixture of discrete Binomial distributions (MDBD). Our sparsification algorithm runs in time $\widetilde{O}(\epsilon^{-2}m_B\log^{3}n\cdot\log^{2}N+\epsilon^{-4}\cdot nN\cdot\log^{4}n\cdot\log^{5}N)$ and exhibits a significant improvement over the result of Cheng et al. [CCLPT15] whose running time is $\widetilde{O}(\epsilon^{-2}\cdot m_B N^2\cdot\mathrm{poly}(\log n))$. Although their algorithm handles general probability distributions, we show that MDBD approximate large class of continuous probability distributions. Moreover, we propose a nearly linear time algorithm that recovers exactly the coefficients of any convex combination of $N+1$ distinct Binomial distributions (MDBD), given as input the target probability distribution $\gamma$ that is induced by a MDBD. In addition, we show that a simple preprocessing step speeds up the run time of Spielman and Peng's [PS14] SDDM Solver from $\widetilde{O}(m_B\log^{3}n\cdot\log^{2}\kappa_B)$ to $\widetilde{O}(m_{B}\log^{2}n + n\log^{4}n\cdot\log^{5}\kappa_B)$, where $\kappa_B$ is the condition number of the input SDDM matrix $B$. When $B$ is a Laplacian matrix, our result leads to a faster algorithm for analyzing the long term behaviour of random walks induced by transition matrices of the form $\sum_{i=0}^{N}\gamma_{i}(D^{-1} A)^i$. We hope that our sparsification algorithm for SDDM matrices can be useful in the context of Machine Learning.
  • Source
    • "One of the contributions of this paper is to show how the calculation of these limits can be reduced to solving a symmetric, diagonally dominant system of linear equations (with different right-hand-sides per community), which can be done in O(m log n) time, where m is the number of edges in the graph. The fact that such systems can be solved in almost linear time was discovered by Spielman and Teng [23] [6] [24] [11] [12] [25]. If we assume that our networks are sparse in the sense that m = O(n), the running time of our algorithm can be bounded by O(n log n). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Complex networks can be typically broken down into groups or modules. Discovering this "community structure" is an important step in studying the large-scale structure of networks. Many algorithms have been proposed for community detection and benchmarks have been created to evaluate their performance. Typically algorithms for community detection either partition the graph (non-overlapping communities) or find node covers (overlapping communities). In this paper, we propose a particularly simple semi-supervised learning algorithm for finding out communities. In essence, given the community information of a small number of "seed nodes", the method uses random walks from the seed nodes to uncover the community information of the whole network. The algorithm runs in time $O(k \cdot m \cdot \log n)$, where $m$ is the number of edges; $n$ the number of links; and $k$ the number of communities in the network. In sparse networks with $m = O(n)$ and a constant number of communities, this running time is almost linear in the size of the network. Another important feature of our algorithm is that it can be used for either non-overlapping or overlapping communities. We test our algorithm using the LFR benchmark created by Lancichinetti, Fortunato, and Radicchi specifically for the purpose of evaluating such algorithms. Our algorithm can compete with the best of algorithms for both non-overlapping and overlapping communities as found in the comprehensive study of Lancichinetti and Fortunato.
Show more


51 Reads
Available from