No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Polynomial acceleration of iterative schemes associated with subproper splittings
Abstract
A subproper splitting of a matrix A is a decomposition A = B − C such that the kernel of A includes that of B while the range of B includes that of A. Our purpose in the present work is to extend the convergence analysis of polynomial acceleration to the case of iterative schemes associated with subproper splittings, in the case of Hermitian matrices and consistent systems. Briefly stated, our conclusions show that the regular theory extends to the subproper case provided that “convergence to the solution of Ax = b” is understood as “convergence to a solution of Ax = b ” while σ(B −1 A) is understood as σ(B+A)\{0} where B+ is the Moore-Penrose inverse of B.
... with b e ~(A) (see below for notation). We have shown in a previous work [19] that such a system can be solved by polynomially accelerated iterative schemes in much the same way as a regular system provided that the splitting (1.2) A = B -C used to define the "preconditioning matrix" B produces a real symmetric positive (semi)definite matrix B with (I. 3) ~ ( B ) c JV(A). The convergence rate then depends on the spectral conditionB+A) plays the same role as x(B-aA) in the regular theory. ...
... Section 5), two alternate cases will occur: either all three matrices U, P, B are regular or they are all singular. In the first case, one can perform the iterations in the same way as in the solution of a regular system (see [19]). In the second ease, one has to use a generalized inverse of B; any {1}-inverse (i.e. ...
... In the second ease, one has to use a generalized inverse of B; any {1}-inverse (i.e. any matrix X such that B X B = B) is convenient for this purpose since, as shown in [19], all { 1}-inverses of B lead to the same convergence rate. In practice it is sufficient to exchange the zero diagonal entries of U in (1.7) for (arbitrary) positive entries to get a nonsingular matrix whose inverse is a { 1 }-inverse of B (cf. Lemma A1 of the Appendix; we assume in this lemma that uu = 0 for some i entails u~j = 0 for all j, the latter property being effectively satisfied by the matrices U considered here, as will be seen in Section 2.) Thus, no practical difficulty arises from the singularity of the preconditioning matrix. ...
Recent works have shown that, whenA is a Stieltjes matrix, its so-called modified incomplete factorizations provide effective preconditioning matrices for solvingAx=b by polynomially accelerated iterative methods. We extend here these results to the singular case with the conclusion that the latter techniques are able to solve singular systems at the same speed as regular systems.
... With these assumptions, we exclude the cases where the system (1.1) is singular, as it may arise in the discrete PDE context when the solution is only defined up to a constant. Actually, we could include such cases on the basis of the results in [18,19,20]. Basically, the kernel of A is then spanned by the constant vector, and, as B has same row-sum, the preconditioner is also singular. ...
... However, it is clear from the results in [19] that, in the context considered here, it will have the same kernel as A. Hence, if the system (1.1) is consistent, the system Bg k = b − Ax k to solve at each iteration is also consistent and one may perform conjugate gradient iterations. The bound (1.3) still holds, where κ(B −1 A) is to be understood as the ratio of the extreme non trivial eigenvalues of the pencil A − νB [18]. Hence, the spectral bounds derived in this paper, which are based on inequalities of the type ν(z, Bz) ≤ (z, Az) ≤ν(z, Bz) ∀z ∈ C n , are readily extended to the case of A, B singular. ...
Considering matrices obtained by the application of a five-point stencil on a 2D rectangular grid, we analyse a preconditioning method introduced by Axelsson and Eijkhout, and by Brand and Heinemann. In this method, one performs a (modified) incomplete factorization with respect to a so-called ‘repeated’ or ‘recursive’ red–black ordering of the unknowns while fill-in is accepted provided that the red unknowns in a same level remain uncoupled.
Considering discrete second order elliptic PDEs with isotropic coefficients, we show that the condition number is bounded by (n) where n is the total number of unknowns (½ log2(√(5) − 1) = 0.153), and thus, that the total arithmetic work for the solution is bounded by (n1.077). Our condition number estimate, which turns out to be better than standard (log2n) estimates for any realistic problem size, is purely algebraic and holds in the presence of Neumann boundary conditions and/or discontinuities in the PDE coefficients.
Numerical tests are reported, displaying the efficiency of the method and the relevance of our analysis.
... When A is a singular weakly diagonally dominant symmetric M-matrix, N = span{e} [14] and the row-sum criterion (3.2) implies that the preconditioner is also singular. The existence analysis proves however then that it is positive semidefinite with N (B) = N (A), which is sufficient for making it a valid preconditioner [21,25]. Moreover, even in such cases ...
... Note that Problem 3 leads to a singular but compatible system. Except possibly DMBILU*, the preconditioners above are then singular too, but admissible because they have same kernel as A [25]. In practice only the last pivot of the pointwise factorization of the last block is zero and it suffices to exchange it for an arbitrary positive value to use these preconditioners trouble free as regular ones [21]. ...
Incomplete factorization preconditioners based on recursive red–black orderings have been shown efficient for discrete second order elliptic PDEs with isotropic coefficients. However, they suffer for some weakness in presence of anisotropy or grid stretching. Here we propose to combine these orderings with block incomplete factorization preconditioning techniques. For implementation considerations, the latter are extended to the case where the block pivots are generalized tridiagonal matrices, say matrices that have at most one nonzero entry per row in their strictly upper triangular part. On the other hand, a new block method is introduced for the improvement of the performance. This method is called IMBILU (improved modified block ILU). Numerical results show that the resulting preconditioner is efficient and robust with respect to both discontinuity and anisotropy in the PDE coefficients. © 1999 Elsevier Science B.V. and IMACS. All rights reserved.
... In his paper, Axelsson [l] proposes a conjugate gradient type method and shows that this method is applicable to the problem considered in the present work. Recently, Notay [16] has given a treatment of polynomial acceleration methods, including steepest descent, conjugate gradients, and Chebyshev acceleration, for the same problem. ...
Consider the linear system of equations Bx = f, where B is an N x N singular matrix, but the system is consistent. In this work we show that iterative techniques coupled with vector extrapolation methods can be used to obtain (approximations to) a solution of Bx = f. We do this by extending the results of some previous work on vector extrapolation methods as they apply to nonsingular matrices B. In particular, we show that the minimal polynomial, reduced rank, and modified minimal polynomial extrapolation methods, and the scalar, topological, and vector epsilon algorithms all produce a solution of Bx = f in at most rank(B) ⩽ N - 1 steps, and that this solution depends on the initial approximation in a simple way. Asymptotic error estimates and error bounds are given for two different limiting procedures that have been considered in previous work. Although we demonstrate all our results for Richardson's iterative method, they are equally valid for any other iterative method.
... It is then easy to see that S , and therefore the preconditioner (5), become necessarily singular too. Although the behavior of iterative solvers for singular systems in now well understood (see Reference [36] for a treatment of singular preconditioners), this raises an additional difficulty since, in fact, we are not interested in solving the singular system, but we want to extrapolate the convergence properties observed in this case to a nearby system, in general non-singular. That is why in the following theorem we assume that the stencil has row-sum ε > 0 , and report on lim ε→0 λ ε , where λ ε denotes an eigenvalue of B −1 ε A ε . ...
Stable finite difference approximations of convection–diffusion equations lead to large sparse linear systems of equations whose coefficient matrix is an M-matrix, which is highly non-symmetric when the convection dominates. For an efficient iterative solution of such systems, it is proposed to consider in the non-symmetric case an algebraic multilevel preconditioning method formerly proposed for pure diffusion problems, and for which theoretical results prove grid independent convergence in this context. These results are supplemented here by a Fourier analysis that applies to constant coefficient problems with periodic boundary conditions whenever using an 'idealized' version of the two-level preconditioner. Within this setting, it is proved that any eigenvalue λ of the preconditioned system satisfies λ −1 − 1 − i c ≤ 1 2 for some real constant c such that |c| ≤ 1 4 . This result holds independently of the grid size and uniformly with respect to the ratio between convection and diffusion. Extensive numerical experiments are conducted to assess the convergence of practical two-and multi-level schemes. These experiments, which include problems with highly variable and rotating convective flow, indicate that the convergence is grid independent. It deteriorates moderately as the convection becomes increasingly dominating, but the convergence factor remains uniformly bounded. This conclusion is supported for both uniform and some non-uniform (stretched) grids.
... We have 0 = ˜ Af = ÂPf from which  = 0 since P is nonsingular, so that (0; f) is an eigenpair of the pencil ( ˜ A; P). All other eigenvectors of ( ˜ A; P) are orthogonal to Pf. Substituting (5.2) in x T ˜ Ax = Âx T Px, for x ∈ span{f} we obtain the lower bound (cf. [27]). We will show below that min ¿ ch 2 as h → 0 with c constant, where h is the mesh characteristic dimension. ...
In this paper, we deal with the problem of solving the large algebraic linear system arising in the numerical solution of a reaction–diffusion (R–D) system associated with myocardial excitation process modeling. We show that an ad hoc preconditioning technique can be devised so as to efficiently and simultaneously handle the differential equations of the R–D system, with no additional memory requirements.Two different formulations are commonly considered for the theoretical and numerical analyses, respectively. We observe that the formulation employed for the theoretical analysis of the problem actually yields the best numerical performance, when compared with the usual numerical scheme.
We improve the conditioning analysis of modified block incomplete factorizations of Stieltjes matrices. Letting N denote the number of diagonal blocks, our results show that the spectral condition number is bounded by N for a large class of two dimensional PDEs.
An upper triangular matrix U = (uij) is said to be ‘S/P consistently ordered’ if for all i, j, i ≠ j, the existe nce of an index k<i, j with uki≠0 and ukj≠0 entails the existence of an index l>i, j such that uil≠0 and ujl≠0. The conditioning analysis of Stieltjes matrices by (unpertubed) modified incomplete factorizations has recently been improved under the assumpion that the approximate upper triangular factor is S/P consistently ordered. Upper eigenvalue bounds provided by previous analyses were critically dependent on the strict diagonal dominance of the upper triangular factor, a dependence which has been reduced to a noncritical one by the new approach. However, in the absence of strict diagonal dominance, the new upper eigenvalue bound is critically dependent on a new parameter, called “maximal reduction ratio”. We investigate here the behaviour of this parameter and we give an algorithm to compute or at least to bound it (from below). We further illustrate how the a priori computation of some lower bound on the maximal reduction ratio may help in choosing the most appropriate factorization strategy.
Beauwens' procedure for obtaining lower eigenvalue bounds for (regular) pencils of matrices A−γB is simplified and extended to the singular case. The theory is then compared, through a particular perturbed modified incomplete factorization, with Notay's generalization of another approach, initiated by Gustafsson, and developed by Axelsson and Barker and by Wilmet.
The use of preconditionings obtained by so called modified incomplete factorizations has become quite popular for the PCG
solution of regular systems arising from the discretization of elliptic PDE's. Our purpose here is to review their recent
extension to the singular case. Because such conditionings may themselves be singular, we first review the extension of the
general theory of polynomial acceleration to the case of singular preconditionings. We emphasize that all results can be formulated
in such a way that they cover both the regular and singular cases. Examples of application are given, displaying the superiority
of the recently developed factorization strategies.
This contribution describes how iterative solvers can meet specific requirements of industrial FE analyses, focusing on the case frequently met where the unknowns are subject to linear equality constraints. Standard iterative methods designed to deal with that kind of problem suffer from a significant overhead with respect to the CPU times involved in the solution of unconstrained problems, whereas the performance of direct solvers traditionally used is not affected. Here we propose a subspace projection method that allows the use of any iterative scheme able to solve the unconstrained problem, with the same preconditioner. We also highlight that there is no loss of efficiency due to the presence of linear constraints.
. A new multilevel preconditioner is proposed for the iterative solution of linear systems whose coefficient matrix is a symmetric M--matrix arising from the discretization of a second order elliptic PDE. It is based on a recursive block incomplete factorization of the matrix partitioned in a two-by-two block form, in which the submatrix related to the fine grid nodes is approximated by a MILU factorization, and the Schur complement computed from a diagonal approximation of the same submatrix. A general algebraic analysis proves optimal order convergence under mild assumptions related to the quality of the approximations of the Schur complement on the one hand, and of the fine grid submatrix on the other hand. This analysis does not require a red black ordering of the fine grid nodes, nor a transformation of the matrices in hierarchical form. Considering more specifically 5 point finite difference approximations of 2D problems, we prove that the spectrum of the preconditioned system is...
Stable finite difference approximations of convection-diffusion equations lead to large sparse linear systems of equations whose coefficient matrix is an M--matrix, which is highly non symmetric when the convection dominates. For an efficient iterative solution of such systems, it is proposed to consider in the non symmetric case an algebraic multilevel preconditioning method formerly proposed for pure diffusion problems, and for which theoretical results prove grid independent convergence in this context. These results are supplemented here by a Fourier analysis which applies to constant coefficient problems with periodic boundary conditions whenever using an "idealized" version of the two-level preconditioner. Within this setting, it is proved that any eigenvalue of the preconditioned system satisfies j Gamma1 Gamma 1 Gamma i cj 1 2 for some real constant c such that jcj 1 4 . This result holds independently of the grid size and uniformly with respect to the ratio betwee...
Complementary subspaces consistency between the linear system and the linear system is defined and studied. This concept is applied to the consideration of iterative schemes for solving (*) obtained through various splittings of the matrix A.
We construct a certain iterative scheme for solving large scale consistent systems of linear equations Ax=b, (*) where A is a complex m X n matrix of rank r, m⩾n, and where A is assumed reasonably well conditioned. The iterative method is obtained through a careful exploitation of an LU-decomposition of A, and, disregarding roundoff errors, it converges to a solution to (*), though not necessarily the minimal l2-norm one, from any starting vector in r iterations. Moreover, once the LU-decomposition of A is complete, only about r3/2 arithmetic operations (multiplications and divisions) are needed to execute the r iterations.
When this book appeared in 1984, the finite element method was already a well-established tool for the analysis of a wide range of problems in science and engineering. Indeed, its ability to model complex geometries in a systematic way put it far ahead of its competitors in many cases. In the years that have passed since then, the method has continued to be the subject of intensive research. Much progress has been made, for example, in the important area of mesh generation. This, combined with techniques for computing error estimates at element level, has led to adaptive meshing, where elements are automatically concentrated where they are needed. Further, work on the p and h-p versions of the finite element method, where p is the degree of the polynomial basis functions and h is the greatest element edge length in the mesh, have made it possible, for some problems, to obtain high accuracy from fairly coarse meshes. Much progress has also been made in the area of mixed finite elements, discontinuous finite elements, mortar finite elements, and in the use of stabilization techniques. Postprocessing techniques, such as stress recovery, have been developed for the extraction of important information from the computed solution. Yet another area of advance has been the topic of structural optimization techniques.
The finite element method transforms a continuous problem to a discrete problem, typically yielding a set of sparse, linear or nonlinear, algebraic set of equations, often with millions of unknowns, that must be solved; hence the efficiency of the finite element method is always linked to the state of the art of numerical linear algebra. Since 1984 much progress has been made also in this area. Although efficient software packages for direct solution methods are available, these tend to be useful only for one- and two-dimensional problems. For three-dimensional problems, preconditioned iterative methods are essential. Multilevel preconditioned based on the two-level methods described in Chapter 7 have been developed for symmetric positive definite problems. The systems arising in structural mechanics can be solved efficiently by using diagonal block matrix preconditioned derived from a separate displacement ordering or, especially in the case of nearly incompressible materials, the 2 × 2 block structure arising from mixed variable methods.
Domain decomposition methods, Schur complement preconditioners, and iterative techniques for mixed and mortar finite element methods have also been developed.
The concept of a regular splitting of a real nonsingular matrix is used in iterative methods for solving linear systems. The purpose of this paper is to extend this concept to rectangular linear systems Ax equals b, in two directions. Firstly, this is accomplished by replacing A** minus **1 by the Moore-Penrose inverse of A, and, secondly, by considering matrices that leave a cone invariant.
Proper splittings of a rectangular matrix A of {\poeratorname{rank}}r are characterized in terms of splittings , where is a nonsingular submatrix of A of order r.It is shown that . Conditions for to converge are given, including monotonicity type conditions under regularity assumptions.
Let H be a Hilbert space, T a bounded linear operator on H into H such that the range of T is closed. Let denote the adjoint of T. In this paper, we establish the convergence of the method of steepest descent to a solution of the equation , for any initial approximation . We also show that the method converges to the unique solution with minimal norm if and only if is in the range of .
A proper splitting of a rectangular matrix A is one of the form A = M − N, where A and M have the same range and null spaces. This concept was introduced by R. Plemmons as a means of generalizing to rectangular and singular matrices the concept of a regular splitting of a nonsingular matrix as introduced by R. Varga. In consideration of the linear system Ax=b, A. Berman and R. Plemmons used a proper splitting of A into M − N and showed that the iteration x(i+1)=M+Nx(i)+M+b converges to A+b, the best least-squares solution to the system, if and only if the spectral radius of M+N is less than one. The purpose of this paper is to further develop the characteristics of proper splittings and to extend these previous results by replacing the Moore-Penrose generalized inverse with a least-squares g-inverse, a minimum-norm g-inverse, or a g-inverse. Also, some criteria are given for comparing convergence rates of Mi−Ni, where A = M1−N1 = M2−N2, and a method is developed for constructing proper splittings of special types of matrices.
In this paper iterative schemes for approximating a solution to a rectangular but consistent linear system Ax = b are studied. Let AϵCm × nr. The splitting A = M − N is called subproper if R(A) ⊆ R(M) and R(A∗) ⊆R(M∗). Consider the iteration . We characterize the convergence of this scheme to a solution of the linear system. When AϵRm×nr, monotonicity and the concept of subproper regular splitting are used to determine a necessary and a sufficient condition for the scheme to converge to a solution.
Iterative methods are discussed for approximating a solution to a singular but consistent square linear systemAx=b. The methods are based upon splittingA=M–N withM nonsingular. Monotonicity and the concept of regular splittings, introduced by Varga, are used to determine some necessary and some sufficient conditions in order that the iterationx
i+1=M–1Nxi+M–1b converge to a solution to the linear system. Finally, applications are given to solving the discrete Neumann problem by iteration which are based upon the inherent monotonicity in the formulation.
Conjugate gradient type methods are discussed for unsymmetric and inconsistent system of equations. For unsymmetric problems, besides conjugate gradient methods based on the normal equations, we also present a (modified) minimal residual (least square) method, which converges for systems with matrices that have a positive definite symmetric part. For inconsistent problems, for completeness we discuss briefly various (well-known) versions of the conjugate gradient method. Preconditioning and rate of convergence are also discussed.
The Theory of Matrices
- L.A. Hageman
- D.M. Young
- L.A. Hageman
- D.M. Young