Nearly-Linear Time Algorithms for Preconditioning and Solving Symmetric, Diagonally Dominant Linear Systems

Source: arXiv

ABSTRACT We present a randomized algorithm that, on input a symmetric, weakly
diagonally dominant n-by-n matrix A with m nonzero entries and an n-vector b,
produces a y such that $\norm{y - \pinv{A} b}_{A} \leq \epsilon \norm{\pinv{A}
b}_{A}$ in expected time $O (m \log^{c}n \log (1/\epsilon)),$ for some constant
c. By applying this algorithm inside the inverse power method, we compute
approximate Fiedler vectors in a similar amount of time. The algorithm applies
subgraph preconditioners in a recursive fashion. These preconditioners improve
upon the subgraph preconditioners first introduced by Vaidya (1990).
For any symmetric, weakly diagonally-dominant matrix A with non-positive
off-diagonal entries and $k \geq 1$, we construct in time $O (m \log^{c} n)$ a
preconditioner B of A with at most $2 (n - 1) + O ((m/k) \log^{39} n)$ nonzero
off-diagonal entries such that the finite generalized condition number
$\kappa_{f} (A,B)$ is at most k, for some other constant c.
In the special case when the nonzero structure of the matrix is planar the
corresponding linear system solver runs in expected time $ O (n \log^{2} n + n
\log n \ \log \log n \ \log (1/\epsilon))$.
We hope that our introduction of algorithms of low asymptotic complexity will
lead to the development of algorithms that are also fast in practice.

1 Bookmark
  • [Show abstract] [Hide abstract]
    ABSTRACT: In this paper, we introduce a new framework for approximately solving flow problems in capacitated, undirected graphs and apply it to provide asymptotically faster algorithms for the maximum $s$-$t$ flow and maximum concurrent multicommodity flow problems. For graphs with $n$ vertices and $m$ edges, it allows us to find an $\epsilon$-approximate maximum $s$-$t$ flow in time $O(m^{1+o(1)}\epsilon^{-2})$, improving on the previous best bound of $\tilde{O}(mn^{1/3} poly(1/\epsilon))$. Applying the same framework in the multicommodity setting solves a maximum concurrent multicommodity flow problem with $k$ commodities in $O(m^{1+o(1)}\epsilon^{-2}k^2)$ time, improving on the existing bound of $\tilde{O}(m^{4/3} poly(k,\epsilon^{-1})$. Our algorithms utilize several new technical tools that we believe may be of independent interest: - We give a non-Euclidean generalization of gradient descent and provide bounds on its performance. Using this, we show how to reduce approximate maximum flow and maximum concurrent flow to the efficient construction of oblivious routings with a low competitive ratio. - We define and provide an efficient construction of a new type of flow sparsifier. In addition to providing the standard properties of a cut sparsifier our construction allows for flows in the sparse graph to be routed (very efficiently) in the original graph with low congestion. - We give the first almost-linear-time construction of an $O(m^{o(1)})$-competitive oblivious routing scheme. No previous such algorithm ran in time better than $\tilde{{\Omega}}(mn)$. We also note that independently Jonah Sherman produced an almost linear time algorithm for maximum flow and we thank him for coordinating submissions.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Spectral clustering is arguably one of the most important algorithms in data mining and machine intelligence; however, its computational complexity makes challenging to use in large scale data analysis. Recently, several approximation algorithms for spectral clustering have been developed in order to alleviate the relevant costs, but theoretical results are lacking. In this paper, we present an approximation algorithm for spectral clustering with strong theoretical evidence of its performance. Our algorithm is based on approximating the eigenvectors of the Laplacian matrix using random projections, a.k.a randomized sketching. Our experimental results demonstrate that the proposed approximation algorithm compares remarkably well to the exact algorithm.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We present the first parallel algorithm for solving systems of linear equations in symmetric, diagonally dominant (SDD) matrices that runs in polylogarithmic time and nearly-linear work. The heart of our algorithm is a construction of a sparse approximate inverse chain for the input matrix: a sequence of sparse matrices whose product approximates its inverse. Whereas other fast algorithms for solving systems of equations in SDD matrices exploit low-stretch spanning trees, our algorithm only requires spectral graph sparsifiers.


Available from