Content uploaded by Stefan Vandewalle
Author content
All content in this area was uploaded by Stefan Vandewalle on Aug 11, 2014
Content may be subject to copyright.
A preview of the PDF is not available
Space-time Concurrent Multigrid Waveform Relaxation
Abstract and Figures
Multigrid waveform relaxation is an algorithm for solving parabolic partial differential equations on multicomputers. It is shown in this pa.per that the algorithm allows a. partitioning of the computational domain into space-time blocks, i.e., subdomains of the space-time grid that a.re treated concurrently by different processors. The space-time concurrent multigrid waveform relaxation method is compared to two methods
that use spatial concurrency only: space-concurrent multigrid waveform relaxation and standard time-stepping. It is illustrated that the use of space-time concurrency enables one to harness the computa.tiona.1 power available on large-sea.le multicomputers. Timing results obtained on an Intel iPSC/2, an Intel iPSC/860 and the Intel Touchstone Delta. a.re presented.
Figures - uploaded by Stefan Vandewalle
Author content
All figure content in this area was uploaded by Stefan Vandewalle
Content may be subject to copyright.
... Parallel Runge Kutta methods were introduced in [20] with good small scale time parallelism. In [19,25], the authors propose to combine multigrid methods with waveform relaxation. Parareal [18] uses a different approach, namely multiple shooting with an approximate Jacobian on a coarse grid, and Parareal techniques led to a new ParaOpt algorithm [9] for optimal control, see also [12]. ...
... Since coth(x) > 1, ∀x > 0, both the numerator and the denominator in (25) are positive, and the difference between them is ...
... Proof. Since we have shown in Corollary 3 that the convergence factor is between 0 and 1 for each eigenvalue d i , we can take (25) and remove the absolute value, ...
We present new Dirichlet-Neumann and Neumann-Dirichlet algorithms with a time domain decomposition applied to unconstrained parabolic optimal control problems. After a spatial semi-discretization, we use the Lagrange multiplier approach to derive a coupled forward-backward optimality system, which can then be solved using a time domain decomposition. Due to the forward-backward structure of the optimality system, three variants can be found for the Dirichlet-Neumann and Neumann-Dirichlet algorithms. We analyze their convergence behavior and determine the optimal relaxation parameter for each algorithm. Our analysis reveals that the most natural algorithms are actually only good smoothers, and there are better choices which lead to efficient solvers. We illustrate our analysis with numerical experiments.
... Strong scaling tests show that the runtime can be significantly reduced by the time-simultaneous multigrid method, achieving a substantial speedup over the sequential approach. Further analysis of the parallel performance of the WRMG method can be found in [8,9]. Additionally, there are some theoretical convergence 2/28 results for the WRMG method. ...
... In this section, we apply a geometric multigrid method in space to the constructed system (8), which is designed to be highly parallelizable on modern hardware architectures. The iterative solver allows parallelization in space while all time steps are treated simultaneously for each spatial grid point. ...
This work focuses on the solution of the convection–diffusion equation, especially for small diffusion coefficients, employing a time-simultaneous multigrid algorithm, which is closely related to multigrid waveform relaxation. For discretization purposes, linear finite elements are used while the Crank–Nicolson scheme acts as the time integrator. By combining all time steps into a global linear system of equations and rearranging the degrees of freedom, a space-only problem is formed with vector-valued unknowns for each spatial node. The generalized minimal residual method with block Jacobi preconditioning can be used to numerically solve the (spatial) problem, allowing a higher degree of parallelization in space. A time-simultaneous multigrid approach is applied, utilizing space-only coarsening and the aforementioned solution techniques for smoothing purposes. Numerical studies analyze the iterative solution technique for 1D test problems. For the heat equation, the number of iterations stays bounded independently of the number of time steps, the time increment, and the spatial resolution. However, convergence issues arise in situations where the diffusion coefficient is small compared to the grid size and the magnitude of the velocity field. Therefore, a higher-order variational multiscale stabilization is used to improve the convergence behavior and solution smoothness without compromising its accuracy in convection-dominated scenarios.
... Their adaptation as a relaxation scheme for space-time multigrid methods was studied extensively in the 1990's, following the first work by Lubich and Ostermann [32]. Much of this work has focused on methods for the heat equation and similar scalar parabolic model problems [25][26][27][39][40][41]. To our knowledge, the only previous application of multigrid waveform relaxation to the Navier-Stokes equations is the work of Oosterlee and Wesseling [35], but this work is limited to a low-order finite-volume discretization and BDF(2) time discretization. ...
... While Hackbusch's original approach simply used the space-time analogues of Jacobi or Gauss-Seidel point relaxation techniques, Lubich and Oostermann quickly proposed the use of so-called waveform relaxation [30] techniques as relaxation methods for space-time multigrid problems [32]. This approach was extensively studied and developed in the 1990's, particularly by Vandewalle and collaborators [25][26][27][39][40][41]. ...
Space-time finite-element discretizations are well-developed in many areas of science and engineering, but much work remains within the development of specialized solvers for the resulting linear and nonlinear systems. In this work, we consider the all-at-once solution of the discretized Navier-Stokes equations over a space-time domain using waveform relaxation multigrid methods. In particular, we show how to extend the efficient spatial multigrid relaxation methods from [37] to a waveform relaxation method, and demonstrate the efficiency of the resulting monolithic Newton-Krylov-multigrid solver. Numerical results demonstrate the scalability of the solver for varying discretization order and physical parameters.
... Nowadays, time-parallel algorithms have seen uptake in a wider range of research fields. Many such parallelin-time methods exist, e.g., [7][8][9][10][11][12][13][14][15][16][17][18] and researchers continue to develop and propose new techniques. On the other hand (and similar to numerical algorithms that fall into other categories than time-parallelism), a growing number of researchers develop convergence analysis for their algorithms to enable well-informed parameter choices and thus improve parallel efficiency, e.g., see [19][20][21][22][23][24][25][26][27][28][29] and many others. ...
... The coarse-grid operator R A 0 P is ideal since one step on the coarse grid is performed by m fine grid time steps, which yields the exact solution with respect to the fine grid. Without approximating the operator R A 0 P, the method would be as expensive, which motivates the approximation in Eq.(8). ...
In this paper, a time-periodic MGRIT algorithm is proposed as a means to reduce the time-to-solution of numerical algorithms by exploiting the time periodicity inherent to many applications in science and engineering. The time-periodic MGRIT algorithm is applied to a variety of linear and nonlinear single- and multiphysics problems that are periodic-in-time. It is demonstrated that the proposed parallel-in-time algorithm can obtain the same time-periodic steady-state solution as sequential time-stepping. It is shown that the required number of MGRIT iterations can be estimated a priori and that the new MGRIT variant can significantly and consistently reduce the time-to-solution compared to sequential time-stepping, irrespective of the number of dimensions, linear or nonlinear PDE models, single-physics or coupled problems and the employed computing resources. The numerical experiments demonstrate that the time-periodic MGRIT algorithm enables a greater level of parallelism yielding faster turnaround, and thus, facilitating more complex and more realistic problems to be solved.
... With this approach, instead of considering one time step at a time, one forms an extended system with the whole set of discrete variables, to which a multigrid method is applied. This increases the efficiency of parallel implementations, as demonstrated in, e.g., [5,9,20]. Various multigrid schemes have been considered, using coarsening in space only, in time only, or simultaneously in spacetime. ...
... In the present paper, we contribute a novel, norm-based, convergence analysis for the case where one uses coarsening in space. In particular, we prove the uniform convergence with respect to both the mesh size and the time step size when using properly defined waveform smoothers [13,15,19,20], while we state accurate, easy-to-asses and rigorous convergence conditions for time-parallel smoothers [10][11][12]19]. Our analysis is, to our knowledge, more rigorous and thorough than previous works about coarsening in space. ...
Space-time multigrid refers to the use of multigrid methods to solve discretized partial differential equations considering at once multiple time steps. A new theoretical analysis is developed for the case where one uses coarsening in space only. It proves bounds on the 2-norm of the iteration matrix that connect it to the norm of the iteration matrix when using the same multigrid method to solve the corresponding stationary problem. When using properly defined wavefront type smoothers, the bound is uniform with respect to the mesh size, the time step size, and the number of time steps, and addresses both the two-grid case and the W-cycle. On the other hand, for time-parallel smoothers, the results clearly show the condition to be satisfied by the time step size to have similar performance as with wavefront type smoothers. The analysis also leads to the definition of an effective smoothing factor that allows one to quickly check the potentialities of a given smoothing scheme. The accuracy of the theoretical estimates is illustrated on a numerical example, highlighting the relevance of the effective smoothing factor and the usefulness in following the provided guidelines to have robustness with respect to the time step size.
... However, both Parareal and PFASST can be perceived as special cases of the multigrid approach [25], as discussed in [27,28]. Besides the multigrid-reduction-in-time (MGRIT) algorithm [29], space-time multigrid methods [30,31] and multigrid wave-form relaxation methods [32,33] have been developed. ...
In this paper we present an approach for multi-level parallel-in-time (PinT) ElectroMagnetic Transient (EMT) simulation. We evaluate the approach in the context of power electronics system level simulation. While PinT approaches to power electronics simulations based on 2-level algorithms have been thoroughly explored in the past, multi-level PinT approaches have not been investigated yet. We use the multigrid reduction in time (MGRIT) method to parallelize a dedicated EMT simulation tool which is capable of switching between different converter models on the fly. The presented approach yields a time-parallel speed-up of up to 10 times compared to the sequential-in-time implementation. We also show that special care has to be taken to synchronize the time grids with the electronic components’ switching periods, indicating that further research into the usage of different models from adequate model-hierarchies is necessary.
... Nowadays, time-parallel algorithms have seen uptake in a wider range of research fields. Many such parallel-in-time methods exist, e.g., [7,8,9,10,11,12,13,14,15,16,17,18] and researchers continue to develop and propose new techniques. On the other hand (and similar to numerical algorithms that fall into other categories than time-parallelism), a growing number of researchers develop convergence analysis for their algorithms to enable well-informed parameter choices and thus improve parallel efficiency, e.g., see [19,20,21,22,23,24,25,26,27,28,29] and many others. ...
In this paper, a time-periodic MGRIT algorithm is proposed as a means to reduce the time-to-solution of numerical algorithms by exploiting the time periodicity inherent to many applications in science and engineering. The time-periodic MGRIT algorithm is applied to a variety of linear and nonlinear single- and multiphysics problems that are periodic-in-time. It is demonstrated that the proposed parallel-in-time algorithm can obtain the same time-periodic steady-state solution as sequential time-stepping. An intuitive convergence criterion is derived and it is shown that the new MGRIT variant can significantly and consistently reduce the time-to-solution compared to sequential time-stepping, irrespective of the number of dimensions, linear or nonlinear PDE models, single-physics or coupled problems and the employed computing resources. The numerical experiments demonstrate that the time-periodic MGRIT algorithm enables a greater level of parallelism yielding faster turnaround, and thus, facilitating more complex and more realistic problems to be solved.
In this work, the convergence behavior of a time-simultaneous two-grid algorithm for the one-dimensional heat equation is studied using Fourier arguments in space. The underlying linear system of equations is obtained by a finite element or finite difference approximation in space while the semi-discrete problem is discretized in time using the θ -scheme. The simultaneous treatment of all time instances leads to a global system of linear equations which provides the potential for a higher degree of parallelization of multigrid solvers due to the increased number of degrees of freedom per spatial unknown.
It is shown that the all-at-once system based on an equidistant discretization in space and time stays well conditioned even if the number of blocked time-steps grows arbitrarily. Furthermore, mesh-independent convergence rates of the considered two-grid algorithm are proved by adopting classical Fourier arguments in space without assuming periodic boundary conditions. The rate of convergence with respect to the Euclidean norm does not deteriorate arbitrarily if the number of blocked time steps increases and, hence, underlines the potential of the solution algorithm under investigation. Numerical studies demonstrate why minimizing the spectral norm of the iteration matrix may be practically more relevant than improving the asymptotic rate of convergence.
We present a time-simultaneous multigrid scheme for parabolic equations that is motivated by blocking multiple time steps together. The resulting method is closely related to multigrid waveform relaxation and is robust with respect to the spatial and temporal grid size and the number of simultaneously computed time steps. We give an intuitive understanding of the convergence behavior and briefly discuss how the theory for multigrid waveform relaxation can be applied in some special cases. Finally, some numerical results for linear and also nonlinear test cases are shown.
We discuss how “parallel-in-space & simultaneous-in-time” Newton-multigrid approaches can be designed which improve the scaling behavior of the spatial parallelism by reducing the latency costs. The idea is to solve many time steps at once and therefore solving fewer but larger systems. These large systems are reordered and interpreted as a space-only problem leading to multigrid algorithm with semi-coarsening in space and line smoothing in time direction. The smoother is further improved by embedding it as a preconditioner in a Krylov subspace method. As a prototypical application, we concentrate on scalar partial differential equations (PDEs) with up to many thousands of time steps which are discretized in time, resp., space by finite difference, resp., finite element methods. For linear PDEs, the resulting method is closely related to multigrid waveform relaxation and its theoretical framework. In our parabolic test problems the numerical behavior of this multigrid approach is robust w.r.t. the spatial and temporal grid size and the number of simultaneously treated time steps. Moreover, we illustrate how corresponding time-simultaneous fixed-point and Newton-type solvers can be derived for nonlinear nonstationary problems that require the described solution of linearized problems in each outer nonlinear step. As the main result, we are able to generate much larger problem sizes to be treated by a large number of cores so that the combination of the robustly scaling multigrid solvers together with a larger degree of parallelism allows a faster solution procedure for nonstationary problems.
The numerical solution of a parabolic partial differential equation is usually calculated by a timestepping method. This precludes the efficient use of vectorization and parallelism if the problem to be solved on each time level is not very large. In this paper an algorithm that overcomes the limitations of the standard marching schemes by solving the problem at all the time levels simultaneously is discussed. The method is applicable to linear and nonlinear problems on arbitrary domains. It can be used to solve initial-boundary value problems as well as time-periodic equations. We have implemented the method on an Intel iPSC/2-VX hypercube. The numerical properties of the method are illustrated by two numerical examples and its performance is compared to that of the best standard solvers.
Let {yn} be a trajectory defined sequentially by a nonlinear vector difference equation yn+1 = Fn+1(yn) with y0 known. A Steffensen-like iterative method is proposed which starts from a guessed sequence {un(0)} for the approximation of {yn} and allows certain computations to be performed in parallel. The sequence {u(k+1)n} is obtained from {u(k)n} by means of a formula of the form u(k+1)n+1 = v(k+1)n+1 + Λ(k+1)n+1(u(k+1)n - u(k)n). Here the vectors v(k+1)n+1 and the matrices Λ(k+1)n+1 each require a function evaluation but can be computed in parallel with respect to n in a suitable interval [Nk, Mk], the length of which depends on the number of processors available. Therefore, for the algorithm to serve effectively, the Steffensen iteration must converge quickly and the machine used must possess a large number of processors. Finally, note that the theory includes the case that the sequence {yn} is given by a one-step ODE solver.
Parabolic and hyperbolic differential equations are often solved numerically by time stepping algorithms. These algorithms have been regarded as sequential in time; that is, the solution on a time level must be known before the computation of the solution at subsequent time levels can start. While this remains true in principle, we demonstrate that it is possible for processors to perform useful work on many time levels simultaneously. Specifically, it is possible for processors assigned to later'' time levels to compute a very good initial guess for the solution based on partial solutions from previous time levels, thus reducing the time required for solution. The reduction in the solution time can be measured as parallel speed up. We demonstrate this algorithm for both linear and nonlinear problems. In addition, we discuss the convergence properties of the method based on the convergence properties of the underlying iterative method, and we present an accurate performance model from which the speedup and other quantities can be estimated. 9 refs., 3 figs., 3 tabs.
Time stepping algorithms are often used to solve parabolic and hyperbolic differential equations numerically. These algorithms are generally regarded as sequential in time; that is, the solution on a time level must be known before the computation of the solution at subsequent time levels can start. While this remains true in principle, we demonstrate that it is possible for processors to perform useful work on many time levels simultaneously. Specifically, it is possible for a processor assigned to a ''later'' time level to compute a very good initial guess for the solution based on approximations to the solutions on ''previous'' time levels, thus reducing the time required for solution. The reduction in the solution time can be measured as parallel speedup. We demonstrate two parallel time stepping algorithms that can be used for both linear and nonlinear problems. We compare the two algorithms and discuss their performance in terms of parameters associated with classical time stepping algorithms. 4 refs., 5 tabs.
We study the method which is obtained when a multi-grid method (in space) is first applied directly to a parabolic intitial-boundary value problem, and discretization in time is done only afterwards. This approach is expected to be well-suited to parallel computation. Further, time marching can be done using different time step-sizes in different parts of the spatial domain.
Standard time-stepping techniques for solving parabolic partial differential equations cannot be parallelized or vectorized efficiently unless the problem to be solved is very large and the number of processors is small. To overcome these limitations several researchers have presented methods that increase the parallel performance by calculating the solution on several time levels simultaneously. In earlier papers we have discussed such a method, based on a combination of the waveform relaxation method and multigrid. In this paper we extend our approach to the case of nonlinear parabolic partial differential equations. We illustrate the application of the new algorithm with numerical examples of initial-boundary-value and time-periodic type and we discuss its implementation on an Intel iPSC/2 hypercube. It is shown that also in the nonlinear case the multigrid waveform relaxation method outperforms the standard sequential solvers.
Waveform methods are a form of generalized Picard iteration developed by electrical engineers for the numerical solution of the large systems of ordinary differential equations that arise in circuit simulation for VLSI design. Their effectiveness is principally due to computational considerations — disk swapping, simple organization of calculations for components with very different frequency spectra and potential use of parallelism. In this paper we first examine the rate of convergence of simple extension of Picard methods suitable for parallel computation. For numerical computation, the differential equation must be replaced by a finite-dimensional equation for an approximation to its solution. The second part of the paper examines the application of the iterative methods directly to finite-difference approximations of the original differential equation. This permits a larger class of methods than obtained by discretizing the differential equations satisfied by successive iterates.
There is no natural parallelism across time in ODEs, so to exploit massive parallelism for a small system of equations it is necessary to use iterative techniques in time, such as waveform. The Picard method, for example, allows each integration to be performed in O(log N) time over N time steps but its convergence is poor for any but almost quadrature problems (∂f∂y small). Generalized Picard, or waveform, may have much faster convergence but less parallelism. This paper considers parallelism across time, explores a proposal made in an earlier paper [2], and reports on some tests made on that method.
The partition method of Wang, for the solution of tridiagonal linear systems, is analysed with regard to data transport between the processors of a parallel (local memory) computer. We propose a modification which reduces the amount of data transport considerably, without affecting the computational complexity and which has about the same degree of parallelism as the original version. We will also discuss the effects of this modification to a generalized version for banded systems. The parallel solution of a bidiagonal system is considered.