[Show abstract][Hide abstract] ABSTRACT: The computational solution of problems can be restricted by the availability of solution methods for linear(ized) systems of equations. In conjunction with iterative methods, preconditioning is often the vital component in enabling the solution of such systems when the dimension is large. We attempt a broad review of preconditioning methods.
[Show abstract][Hide abstract] ABSTRACT: The solution of quadratic or locally quadratic extremum problems subject to linear(ized) constraints gives rise to linear systems in saddle point form. This is true whether in the continuous or the discrete setting, so saddle point systems arising from the discretization of partial differential equation problems, such as those describing electromagnetic problems or incompressible flow, lead to equations with this structure, as do, for example, interior point methods and the sequential quadratic programming approach to nonlinear optimization. This survey concerns iterative solution methods for these problems and, in particular, shows how the problem formulation leads to natural preconditioners which guarantee a fast rate of convergence of the relevant iterative methods. These preconditioners are related to the original extremum problem and their effectiveness - in terms of rapidity of convergence - is established here via a proof of general bounds on the eigenvalues of the preconditioned saddle point matrix on which iteration convergence depends.
[Show abstract][Hide abstract] ABSTRACT: Circulant preconditioning for symmetric Toeplitz linear systems is well established; theoretical guarantees of fast convergence for the conjugate gradient method are descriptive of the convergence seen in computations. This has led to robust and highly efficient solvers based on use of the fast Fourier transform exactly as originally envisaged in [G. Strang, Stud. Appl. Math., 74 (1986), pp. 171-176]. For nonsymmetric systems, the lack of generally descriptive convergence theory for most iterative methods of Krylov type has provided a barrier to such a comprehensive guarantee, though several methods have been proposed and some analysis of performance with the normal equations is available. In this paper, by the simple device of reordering, we rigorously establish a circulant preconditioned short recurrence Krylov subspace iterative method of minimum residual type for nonsymmetric (and possibly highly nonnormal) Toeplitz systems. Convergence estimates similar to those in the symmetric case are established.
SIAM Journal on Matrix Analysis and Applications 01/2015; 36(1):273-288. DOI:10.1137/140974213 · 1.59 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: This paper focuses on the spectral distribution of kernel matrices related to radial basis functions. The asymptotic behaviour of eigenvalues of kernel matrices related to radial basis functions with different smoothness are studied. These results are obtained by estimated the coefficients of an orthogonal expansion of the underlying kernel function. Beside many other results, we prove that there are exactly k+d−1 d−1 eigenvalues in the same order for analytic separable kernel functions like the Gaussian in R d . This gives theoretical support for how to choose the diagonal scaling matrix in the RBF-QR method (Fornberg et al, SIAM J. Sci. Comput. (33), 2011) which can stably compute Gaussian radial basis function interpolants.
[Show abstract][Hide abstract] ABSTRACT: Saddle-point systems arise in many applications areas, in fact in any situation where extremum principle arises with constraints. The Stokes problem describing slow viscous flow of an incompressible fluid is an example coming form partial differential equations and in the area of Optimization such problems are ubiquitous. In this manuscript we show how new approaches for the solution of saddle-point systems arising in Optimization can be derived from the Bramble-Pasciak Conjugate Gradient approach widely used in PDEs and more recent generalizations thereof. In particular we derive a class of new solution methods based on the use of Preconditioned Conjugate Gradients in non-standard inner products and demonstrate how these can be understood through more standard machinery. We show connections to Constraint Preconditioning and give the results of numerical computations on a number of standard Optimization test examples.
[Show abstract][Hide abstract] ABSTRACT: For a prescribed porosity, the coupled magma/mantle flow equations can be
formulated as a two field system of equations with velocity and pressure
unknowns. Previous work has shown that while optimal preconditioners for the
two field formulation can be constructed, the construction of preconditioners
that are uniform with respect to model parameters is difficult. This limits the
applicability of two field preconditioners in certain regimes of practical
interest. We address this issue by reformulating the governing equations as a
three field problem, which removes a term that was problematic in the two field
formulation in favour of an additional equation for a pressure-like field. For
the three-field problem, we develop and analyse new preconditioners and we show
numerically that the new three-field preconditioners are optimal in terms of
problem size and less sensitive to model parameters compared to the two-field
preconditioner. This extends the applicability of optimal preconditioners for
coupled mantle/magma dynamics into parameter regimes of important physical
interest.
[Show abstract][Hide abstract] ABSTRACT: It is known that interpolation with radial basis functions of the same shape can guarantee a nonsingular interpolation matrix, whereas little was known when one uses various shapes. In this paper, we prove that functions from a class of compactly supported radial basis functions are convex on a certain region; based on this local convexity and other local geometrical properties of the interpolation points, we construct a sufficient condition which guarantees diagonally dominant interpolation matrices for radial basis functions interpolation with different shapes. The proof is constructive and can be used to design algorithms directly. Numerical examples show that the scheme has a low accuracy but can be implemented efficiently. It can be used for inaccurate models where efficiency is more desirable. Large scale 3D implicit surface reconstruction problems are used to demonstrate the utility and reasonable results can be obtained efficiently.
Journal of Scientific Computing 10/2014; DOI:10.1007/s10915-014-9919-9 · 1.70 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: We consider the convergence of the algorithm GMRES of Saad and Schultz for solving linear equations Bx=b, where B ∈ Cn × n is nonsingular and diagonalizable, and b ∈ Cn. Our analysis explicitly includes the initial residual vector r0. We show that the GMRES residual norm satisfies a weighted polynomial least-squares problem on the spectrum of B, and that GMRES convergence reduces to an ideal GMRES problem on a rank-1 modification of the diagonal matrix of eigenvalues of B. Numerical experiments show that the new bounds can accurately describe GMRES convergence.
IMA Journal of Numerical Analysis 04/2014; 34(2). DOI:10.1093/imanum/drt025 · 1.70 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Mastronardi and Van Dooren [SIAM J. Matrix Anal. Appl., 34 ( 2013), pp. 173-196] recently introduced the block antitriangular ("Batman") decomposition for symmetric indefinite matrices. Here we show the simplification of this factorization for saddle point matrices and demonstrate how it represents the common nullspace method. We show that rank-1 updates to the saddle point matrix can be easily incorporated into the factorization and give bounds on the eigenvalues of matrices important in saddle point theory. We show the relation of this factorization to constraint preconditioning and how it transforms but preserves the structure of block diagonal and block triangular preconditioners.
SIAM Journal on Matrix Analysis and Applications 04/2014; 35(2):339-353. DOI:10.1137/130934933 · 1.59 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: The solution of Cahn-Hilliard variational inequalities is of interest in many applications. We discuss the use of them as a tool for binary image inpainting. This has been done before using double-well potentials but not for nonsmooth potentials as considered here. The existing bound constraints are incorporated via the Moreau-Yosida regularization technique. We develop effective preconditioners for the efficient solution of the Newton steps associated with the fast solution of the Moreau-Yosida regularized problem. Numerical results illustrate the efficiency of our approach. Moreover, precise eigenvalue intervals are given for the preconditioned system using a double-well potential. A comparison between the smooth and nonsmooth Cahn-Hilliard inpainting models shows that the latter achieves better results.
[Show abstract][Hide abstract] ABSTRACT: This article considers the iterative solution of a finite element
discretisation of magma dynamics equations. In simplified form, the magma
dynamics equations share some features of the Stokes equations. We therefore
formulate, analyse and numerically test a Elman, Silvester and Wathen-type
block preconditioner for magma dynamics. We prove analytically, and demonstrate
numerically, optimality of the preconditioner. The presented analysis
highlights the dependence of the preconditioner on parameters in the magma
dynamics equations that can affect convergence of iterative linear solvers. The
analysis is verified through a range of two- and three-dimensional numerical
examples on unstructured grids, from simple illustrate problems through to
large problems on subduction zone-like geometries. The computer code to
reproduce all numerical examples is freely available as supporting material.
[Show abstract][Hide abstract] ABSTRACT: Amongst recent contributions to preconditioning methods for saddle point sys-tems, standard iterative methods in nonstandard inner products have been use-fully employed. Krzy˙ zanowski (Numer. Linear Algebra Appl. 2011; 18:123–140) identified a two-parameter family of preconditioners in this context and Stoll and Wathen (SIAM J. Matrix Anal. Appl. 2008; 30:582–608) introduced combination preconditioning, where two preconditioners, self-adjoint with respect to different inner products, can lead to further preconditioners and associated bilinear forms or inner products. Preconditioners that render the preconditioned saddle point matrix nonsymmetric but self-adjoint with respect to a nonstandard inner prod-uct always allow a MINRES-type method (W-PMINRES) to be applied in the relevant inner product. If the preconditioned matrix is also positive definite with respect to the inner product a more efficient CG-like method (W-PCG) can be reliably used. We establish eigenvalue expressions for Krzy˙ zanowski precondition-ers and show that for a specific choice of parameters, although the Krzy˙ zanowski preconditioned saddle point matrix is self-adjoint with respect to an inner product, it is never positive definite. We provide explicit expressions for the combination of certain preconditioners and prove the rather counterintuitive result that the combination of two specific preconditioners for which only W-PMINRES can be reliably used leads to a preconditioner for which, for certain parameter choices, W-PCG is reliably applicable. That is, combining two indefinite preconditioners can lead to a positive definite preconditioner. This combination preconditioner out-performs either of the two preconditioners from which it is formed for a number of test problems.
Numerical Linear Algebra with Applications 10/2013; 20(5). DOI:10.1002/nla.1843 · 1.32 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: We consider the solution of left preconditioned linear systems P−1Cx=P−1cP−1Cx=P−1c, where P,C∈Cn×nP,C∈Cn×n are non-Hermitian, c∈Cnc∈Cn, and CC, PP, and P−1CP−1C are diagonalisable with spectra symmetric about the real line. We prove that, when PP and CC are self-adjoint with respect to the same Hermitian sesquilinear form, the convergence of a minimum residual method in a particular nonstandard inner product applied to the preconditioned linear system is bounded by a term that depends only on the spectrum of P−1CP−1C. The inner product is related to the spectral decomposition of PP. When PP is self-adjoint with respect to a nearby Hermitian sesquilinear form to CC, the convergence of a minimum residual method in this nonstandard inner product applied to the preconditioned linear system is bounded by a term involving the eigenvalues of P−1CP−1C and a constant factor. The size of this factor is related to the nearness of the Hermitian sesquilinear forms. Numerical experiments indicate that for certain matrices eigenvalue-dependent convergence is observed both for the nonstandard method and for standard GMRES.
Journal of Computational and Applied Mathematics 09/2013; 249:57–68. DOI:10.1016/j.cam.2013.02.020 · 1.27 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: In this manuscript, we describe effective solvers for the optimal control of stabilized convection-diffusion control problems. We employ the local projection stabilization, which results in the same matrix system whether the discretize-then-optimize or optimize-then-discretize approach for this problem is used. We then derive two effective preconditioners for this problem, the first to be used with MINRES and the second to be used with the Bramble-Pasciak Conjugate Gradient method. The key components of both preconditioners are an accurate mass matrix approximation, a good approximation of the Schur complement, and an appropriate multigrid process to enact this latter approximation. We present numerical results to illustrate that these preconditioners result in convergence in a small number of iterations, which is robust with respect to the step-size h and the regularization parameter β for a range of problems.
[Show abstract][Hide abstract] ABSTRACT: In this article, we motivate, derive and test effective preconditioners to be used with the Minres algorithm for solving a number of saddle point systems, which arise in PDE constrained optimization problems. We consider the distributed control problem involving the heat equation with two different functionals, and the Neumann boundary control problem involving Poisson's equation and the heat equation. Crucial to the effectiveness of our preconditioners in each case is an effective approximation of the Schur complement of the matrix system. In each case, we state the problem being solved, propose the preconditioning approach, prove relevant eigenvalue bounds, and provide numerical results which demonstrate that our solvers are effective for a wide range of regularization parameter values, as well as mesh sizes and time-steps.
SIAM Journal on Matrix Analysis and Applications 10/2012; 33(4). DOI:10.1137/110847949 · 1.59 Impact Factor