[Show abstract][Hide abstract] ABSTRACT: Saddle-point systems arise in many applications areas, in fact in any situation where extremum principle arises with constraints. The Stokes problem describing slow viscous flow of an incompressible fluid is an example coming form partial differential equations and in the area of Optimization such problems are ubiquitous. In this manuscript we show how new approaches for the solution of saddle-point systems arising in Optimization can be derived from the Bramble-Pasciak Conjugate Gradient approach widely used in PDEs and more recent generalizations thereof. In particular we derive a class of new solution methods based on the use of Preconditioned Conjugate Gradients in non-standard inner products and demonstrate how these can be understood through more standard machinery. We show connections to Constraint Preconditioning and give the results of numerical computations on a number of standard Optimization test examples.
[Show abstract][Hide abstract] ABSTRACT: It is known that interpolation with radial basis functions of the same shape can guarantee a nonsingular interpolation matrix, whereas little was known when one uses various shapes. In this paper, we prove that functions from a class of compactly supported radial basis functions are convex on a certain region; based on this local convexity and other local geometrical properties of the interpolation points, we construct a sufficient condition which guarantees diagonally dominant interpolation matrices for radial basis functions interpolation with different shapes. The proof is constructive and can be used to design algorithms directly. Numerical examples show that the scheme has a low accuracy but can be implemented efficiently. It can be used for inaccurate models where efficiency is more desirable. Large scale 3D implicit surface reconstruction problems are used to demonstrate the utility and reasonable results can be obtained efficiently.
Journal of Scientific Computing 10/2014; · 1.71 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: The solution of Cahn-Hilliard variational inequalities is of interest in many applications. We discuss the use of them as a tool for binary image inpainting. This has been done before using double-well potentials but not for nonsmooth potentials as considered here. The existing bound constraints are incorporated via the Moreau-Yosida regularization technique. We develop effective preconditioners for the efficient solution of the Newton steps associated with the fast solution of the Moreau-Yosida regularized problem. Numerical results illustrate the efficiency of our approach. Moreover, precise eigenvalue intervals are given for the preconditioned system using a double-well potential. A comparison between the smooth and nonsmooth Cahn-Hilliard inpainting models shows that the latter achieves better results.
[Show abstract][Hide abstract] ABSTRACT: Amongst recent contributions to preconditioning methods for saddle point sys-tems, standard iterative methods in nonstandard inner products have been use-fully employed. Krzy˙ zanowski (Numer. Linear Algebra Appl. 2011; 18:123–140) identified a two-parameter family of preconditioners in this context and Stoll and Wathen (SIAM J. Matrix Anal. Appl. 2008; 30:582–608) introduced combination preconditioning, where two preconditioners, self-adjoint with respect to different inner products, can lead to further preconditioners and associated bilinear forms or inner products. Preconditioners that render the preconditioned saddle point matrix nonsymmetric but self-adjoint with respect to a nonstandard inner prod-uct always allow a MINRES-type method (W-PMINRES) to be applied in the relevant inner product. If the preconditioned matrix is also positive definite with respect to the inner product a more efficient CG-like method (W-PCG) can be reliably used. We establish eigenvalue expressions for Krzy˙ zanowski precondition-ers and show that for a specific choice of parameters, although the Krzy˙ zanowski preconditioned saddle point matrix is self-adjoint with respect to an inner product, it is never positive definite. We provide explicit expressions for the combination of certain preconditioners and prove the rather counterintuitive result that the combination of two specific preconditioners for which only W-PMINRES can be reliably used leads to a preconditioner for which, for certain parameter choices, W-PCG is reliably applicable. That is, combining two indefinite preconditioners can lead to a positive definite preconditioner. This combination preconditioner out-performs either of the two preconditioners from which it is formed for a number of test problems.
Numerical Linear Algebra with Applications 10/2013; 20(5). · 1.20 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: We consider the solution of left preconditioned linear systems P−1Cx=P−1cP−1Cx=P−1c, where P,C∈Cn×nP,C∈Cn×n are non-Hermitian, c∈Cnc∈Cn, and CC, PP, and P−1CP−1C are diagonalisable with spectra symmetric about the real line. We prove that, when PP and CC are self-adjoint with respect to the same Hermitian sesquilinear form, the convergence of a minimum residual method in a particular nonstandard inner product applied to the preconditioned linear system is bounded by a term that depends only on the spectrum of P−1CP−1C. The inner product is related to the spectral decomposition of PP. When PP is self-adjoint with respect to a nearby Hermitian sesquilinear form to CC, the convergence of a minimum residual method in this nonstandard inner product applied to the preconditioned linear system is bounded by a term involving the eigenvalues of P−1CP−1C and a constant factor. The size of this factor is related to the nearness of the Hermitian sesquilinear forms. Numerical experiments indicate that for certain matrices eigenvalue-dependent convergence is observed both for the nonstandard method and for standard GMRES.
Journal of Computational and Applied Mathematics 09/2013; 249:57–68. · 0.99 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: In this manuscript, we describe effective solvers for the optimal control of stabilized convection-diffusion control problems. We employ the local projection stabilization, which results in the same matrix system whether the discretize-then-optimize or optimize-then-discretize approach for this problem is used. We then derive two effective preconditioners for this problem, the first to be used with MINRES and the second to be used with the Bramble-Pasciak Conjugate Gradient method. The key components of both preconditioners are an accurate mass matrix approximation, a good approximation of the Schur complement, and an appropriate multigrid process to enact this latter approximation. We present numerical results to illustrate that these preconditioners result in convergence in a small number of iterations, which is robust with respect to the step-size h and the regularization parameter β for a range of problems.
[Show abstract][Hide abstract] ABSTRACT: In this article, we motivate, derive and test effective preconditioners to be used with the Minres algorithm for solving a number of saddle point systems, which arise in PDE constrained optimization problems. We consider the distributed control problem involving the heat equation with two different functionals, and the Neumann boundary control problem involving Poisson's equation and the heat equation. Crucial to the effectiveness of our preconditioners in each case is an effective approximation of the Schur complement of the matrix system. In each case, we state the problem being solved, propose the preconditioning approach, prove relevant eigenvalue bounds, and provide numerical results which demonstrate that our solvers are effective for a wide range of regularization parameter values, as well as mesh sizes and time-steps.
SIAM Journal on Matrix Analysis and Applications 10/2012; 33(4). · 1.34 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Solving problems regarding the optimal control of partial differential equations (PDEs)—also known as PDE-constrained optimization—is a frontier area of numerical analysis. Of particular interest is the problem of flow control, where one would like to effect some desired flow by exerting, for example, an external force. The bottleneck in many current algorithms is the solution of the optimality system—a system of equations in saddle point form that is usually very large and ill conditioned. In this paper we describe two preconditioners—a block diagonal preconditioner for the minimal residual method and a block lower-triangular preconditioner for a nonstandard conjugate gradient method—which can be effective when applied to such problems where the PDEs are the Stokes equations. We consider only distributed control here, although we believe other problems could be treated in the same way. We give numerical results, and we compare these with those obtained by solving the equivalent forward problem using similar techniques.
SIAM J. Scientific Computing. 01/2011; 33:2903-2926.
[Show abstract][Hide abstract] ABSTRACT: This paper proposes a new preconditioning scheme for a linear system with a saddle-point structure arising from a hybrid approximation scheme on the sphere, an approximation scheme that combines (local) spherical radial basis functions and (global) spherical polynomials. Making use of a recently derived inf-sup condition  and the Brezzi stability and convergence theorem for this approximation scheme, we show that the linear system can be optimally preconditioned with a suitable block-diagonal preconditioner. Numerical experiments with a non-uniform distribution of data points support the theoretical conclusions. Comment: 14 pages, 1 figure, submitted
[Show abstract][Hide abstract] ABSTRACT: Existing convergence bounds for Krylov subspace methods such as GM-RES for nonsymmetric linear systems give little mathematical guidance for the choice of preconditioner. Here, we establish a desirable mathematical property of a preconditioner which guarantees that convergence of a mini-mum residual method will essentially depend only on the eigenvalues of the preconditioned system, as is true in the symmetric case. Our theory covers only a subset of nonsymmetric coefficient matrices but computations indicate that it might be more generally applicable.
[Show abstract][Hide abstract] ABSTRACT: Optimization problems with constraints which require the solution of a partial differential equation arise widely in many areas of the sciences and engineering, particularly in problems of design. The solution of such PDE-constrained optimization problems is usually a major computational task. Here we consider simple problems of this type: distributed control problems in which the 2- and 3-dimensional Poisson problem is the PDE. The large-dimensional linear systems which result from discretization and which need to be solved are of saddle-point type. We introduce two optimal preconditioners for these systems, which lead to convergence of symmetric Krylov subspace iterative methods in a number of iterations which does not increase with the dimension of the discrete problem. These preconditioners are block structured and involve standard multigrid cycles. The optimality of the preconditioned iterative solver is proved theoretically and verified computationally in several test cases. The theoretical proof indicates that these approaches may have much broader applicability for other PDEs.
SIAM J. Scientific Computing. 01/2010; 32:271-298.
[Show abstract][Hide abstract] ABSTRACT: Saddle-point systems arise in many applications areas, in fact in any situation where an extremum principle arises with constraints. The Stokes problem describing slow viscous flow of an incompressible fluid is a classic example coming from PDEs and in the area of optimization such problems are ubiquitous. In this paper we present a framework into which many well-known methods for solving saddle-point systems fit. Based on this description we show how new approaches for the solution of saddle-point systems arising in optimization can be derived from the Bramble-Pasciak conjugate gradient approach widely used in PDEs and more recent generalizations thereof. In particular we derive a class of new solution methods based on the use of preconditioned conjugate gradients in nonstandard inner products and demonstrate how these can be understood through more standard machinery. We show connections to constraint preconditioning and give the results of numerical computations on a number of standard optimization test examples.
SIAM J. Scientific Computing. 01/2010; 32:249-270.
[Show abstract][Hide abstract] ABSTRACT: We analyze pressure stabilized finite element methods for the solution of the generalized Stokes problem and investigate their stability and convergence properties. An important feature of the method is that the pressure gradient unknowns can be eliminated locally thus leading to a decoupled system of equations. Although stability of the method has been established, for the homogeneous Stokes equations, the proof given here is based on the existence of a special interpolant with additional orthogonal property with respect to the projection space. This, makes it a lot simpler and more attractive. The resulting stabilized method is shown to lead to optimal rates of convergence for both velocity and pressure approximations.
Computer Methods in Applied Mechanics and Engineering 01/2009; 198(5):877-883. · 2.62 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: Saddle point problems arise frequently in many applications in science and engineering, including constrained optimization,
mixed finite element formulations of partial differential equations, circuit analysis, and so forth. Indeed the formulation
of most problems with constraints gives rise to saddle point systems. This paper provides a concise overview of iterative
approaches for the solution of such systems which are of particular importance in the context of large scale computation.
In particular we describe some of the most useful preconditioning techniques for Krylov subspace solvers applied to saddle
point problems, including block and constraint preconditioners.