Content uploaded by Mohamed Kamel RIAHI

Author content

All content in this area was uploaded by Mohamed Kamel RIAHI on Oct 24, 2015

Content may be subject to copyright.

A NEW APPROACH TO IMPROVE

ILL-CONDITIONED PARABOLIC OPTIMAL

CONTROL PROBLEM VIA TIME DOMAIN

DECOMPOSITION

Mohamed Kamel Riahi

To cite this version:

Mohamed Kamel Riahi. A NEW APPROACH TO IMPROVE ILL-CONDITIONED

PARABOLIC OPTIMAL CONTROL PROBLEM VIA TIME DOMAIN DECOMPOSITION.

2015. <hal-00974285v2>

HAL Id: hal-00974285

https://hal.inria.fr/hal-00974285v2

Submitted on 14 Jan 2015

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of sci-

entiﬁc research documents, whether they are pub-

lished or not. The documents may come from

teaching and research institutions in France or

abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est

destin´ee au d´epˆot et `a la diﬀusion de documents

scientiﬁques de niveau recherche, publi´es ou non,

´emanant des ´etablissements d’enseignement et de

recherche fran¸cais ou ´etrangers, des laboratoires

publics ou priv´es.

A NEW APPROACH TO IMPROVE ILL-CONDITIONED PARABOLIC

OPTIMAL CONTROL PROBLEM VIA TIME DOMAIN DECOMPOSITION

Mohamed Kamel RIAHI∗1

1Department of mathematical science, New Jersey Institute of Technology, University Heights

Newark, New Jersey, USA.

January 14, 2015

Abstract. In this paper we present a new steepest-descent type algorithm for convex op-

timization problems. Our algorithm pieces the unknown into sub-blocs of unknowns and

considers a partial optimization over each sub-bloc. In quadratic optimization, our method

involves Newton technique to compute the step-lengths for the sub-blocs resulting descent di-

rections. Our optimization method is fully parallel and easily implementable, we ﬁrst presents

it in a general linear algebra setting, then we highlight its applicability to a parabolic optimal

control problem, where we consider the blocs of unknowns with respect to the time dependency

of the control variable. The parallel tasks, in the last problem, turn“on” the control during

a speciﬁc time-window and turn it “oﬀ” elsewhere. We show that our algorithm signiﬁcantly

improves the computational time compared with recognized methods. Convergence analysis of

the new optimal control algorithm is provided for an arbitrary choice of partition. Numerical

experiments are presented to illustrate the eﬃciency and the rapid convergence of the method.

Steepest descent method, Newton method, ill conditioned Optimal control, time domain de-

composition.

1. Introduction

Typically the improvement of iterative methods is based on an implicit transformation of the

original linear system in order to get a new system which has a condition number ideally close to

one see [11,13,25] and references therein. This technique is known as preconditioning. Modern

preconditioning techniques such as algebraic multilevel e.g. [20,24] and domain decomposition

methods e.g. [23,27,4,15] attempt to produce eﬃcient tools to accelerate convergence. Other

techniques have introduced a diﬀerent deﬁnition of the descent directions, for example, CG-

method, GMRES, FGMRES, BFGS, or its limited memory version l-BFGS see for instance [25].

Others approaches (e.g. [5],[12] and [28] without being exhaustive) propose diﬀerent formulas for

the line-search in order to enhance the optimization procedure.

The central investigation of this paper is the enhancement of the iterations of the steepest

descent algorithm via an introduction of a new formulation for the line-search. Indeed, we show

how to achieve an optimal vectorized step-length for a given set of descent directions. Steepest

descent methods [7] are usually used for solving, for example, optimization problems, control

with partial diﬀerential equations (PDEs) constraints and inverse problems. Several approaches

have been developed in the cases of constrained and unconstrained optimization.

It is well-known that the algorithm has a slow convergence rate with ill-conditioned problems

because the number of iterations is proportional to the condition number of the problem. The

method of J.Barzila and J.Borwein [2] based on two-point step-length for the steepest-descent

method for approximating the secant equation avoids this handicap. Our method is very diﬀerent

∗Mohamed Kamel RIAHI : riahi@njit.edu http://web.njit.edu/~riahi

1

2 MULTI-STEEPEST DESCENT ALGORITHM

because ﬁrst, it is based on a decomposition of the unknown and proposes a set of bloc descent

directions, and second because it is general where it can be coupled together with any least-

square-like optimization procedure.

The theoretical basis of our approach is presented and applied to the optimization of a positive

deﬁnite quadratic form. Then we apply it on a complex engineering problem involving control of

system governed by PDEs. We consider the optimal heat control which is known to be ill-posed

in general (and well-posed under some assumptions) and presents some particular theoretical

and numerical challenges. We handle the ill-posedness degree of the heat control problem by

varying the regularization parameter and apply our methods in the handled problem to show the

eﬃciency of our algorithm. The distributed- and boundary-control cases are both considered.

This paper is organized as follows: In Section 2, we present our method in a linear algebra

framework to highlight its generality. Section 3 is devoted to the introduction of the optimal

control problem with constrained PDE on which we will apply our method. We present the

Euler-Lagrange-system associated to the optimization problem and give the explicit formulation

of the gradient in both cases of distributed- and boundary-control. Then, we present and explain

the parallel setting for our optimal control problem. In Section 4, we perform the convergence

analysis of our parallel algorithm. In Section 5, we present the numerical experiments that

demonstrate the eﬃciency and the robustness of our approach. We make concluding remarks in

Section 6. For completeness, we include calculus results in the Appendix.

Let Ω be a bounded domain in R3, and Ωc⊂Ω, the boundary of Ω is denoted by ∂Ω. We

denote by Γ ⊂∂Ω a part of this boundary. We denote h., .i2(respectively h., .icand h., .iΓ) the

standard L2(Ω) (respectively L2(Ωc) and L2(Γ)) inner-product that induces the L2(Ω)-norm k.k2

on the domain Ω (respectively k·kcon Ωcand k·kΓon Γ).

In the case of ﬁnite dimensional vector space in Rm, the scalar product aTbof aand b(where

aTstands for the transpose of a) is denoted by h., .i2too. The scalar product with respect to

the matrix A, i.e. hx, Axi2is denote by hx, xiAand its induced norm is denoted by kxkA. The

transpose of the operator Ais denoted by AT. The Hilbert space L2(0, T ;L2(Ωc)) (respectively

L2(0, T ;L2(Γ))) is endowed by the scalar product h., .ic,I ( respectively h., .iΓ,I) that induces the

norm k.kc,I (respectively k.kΓ,I ).

2. Enhanced steepest descent iterations

The steepest descent algorithm minimizes at each iteration the quadratic function q(x) =

kx−x?k2

A, where Ais assumed to be a symbiotic positive deﬁnite (SPD) matrix and x?is the

minimum of q. The vector −∇q(x) is locally the descent direction that yields the fastest rate of

decrease of the quadratic form q. Therefore all vectors of the form x+θ∇q(x), where θis a suitable

negative real value, minimize q. The choice of θis found by looking for the mins<0q(x+s∇q(x))

with the use of a line-search technique. In the case where qis a quadratic form θis given by

−k∇q(x)k2

2/k∇q(x)k2

A. We recall in Algorithm 1the steepest descent algorithm; Convergence

is a boolean variable based on estimation of the residual vector rk< , where is the stopping

criterion.

Our method proposes to modify the stepe 5.of Algorithm 1. It considers the step-length

θ∈R−\{0}as a vector in Rˆn

−\{0}where ˆnis an integer such that 1 ≤ˆn≤size(x), we shall

denote this new vector as Θˆn.

In the following, it is assumed that for a giving vector x∈Rm, the integer ˆndivides mwith

null rest. In this context, let us introduce the identity operators IRmwitch is an m-by-mmatrix

and its partition (partition of unity) given by the projection operators {πn}ˆn

n=1 : projectors from

MULTI-STEEPEST DESCENT ALGORITHM 3

Algorithm 1: Steepest descent.

Input:x0;

1k= 0;

2while Convergence do

3rk=∇qk:= ∇q(xk);

4Compute Ark;

5Compute θk=−krkk2

2/krkk2

A;

6xk+1 =xk+θkrk;

7k=k+ 1;

8end

Rminto a set of canonical basis {ei}i. These operators are deﬁned for 1 ≤n≤ˆnby

πn:Rm→Rm

ˆn

x7→ πn(x) =

n×m

ˆn

X

i=(n−1)×m

ˆn+1

hei, xi2ei.

For reading conveniences, we deﬁne ˜xna vector in Rmsuch that ˜xn:= πn(x). The concatenation

of ˜xnfor all 1 ≤n≤ˆnis denoted by

ˆxˆn=

ˆn

M

n=1

πn(x) =

ˆn

M

n=1

˜xn∈Rm.

We remark that πnsatisfy Lˆn

n=1 πn=IRm.

Recall the gradient ∇x= ( ∂

∂x1,..., ∂

∂xm)T, and deﬁne the bloc gradient ∇ˆxˆn=∇T

˜x1,...,∇T

˜xˆnT,

where obviously ∇T

˜xn= ( ∂

∂x(n−1)×m

ˆn+1 ,..., ∂

∂xn×m

ˆn

)T. In the spirit of this decomposition we in-

vestigate, in the sequel, the local descent directions as the bloc partial derivatives with respect to

the bloc-variables (˜xn)n=ˆn

n=1 . We aim, therefore, at ﬁnding Θˆn= (θ1, . . . , θ ˆn)T∈Rˆnthat ensures

the min(θn)n<0qˆxk

ˆn+Lˆn

n=1 θn∇˜xnq(ˆxk

ˆn).

We state hereafter a motivating result, which its proof is straightforward because the spaces

are embedded. Let us ﬁrst, denote by

(1) Φˆn(Θˆn) : Rˆn→R+

Θˆn7→ qˆxˆn+Lˆn

n=1 θn∇˜xnq(ˆxˆn)

which is quadratic because qis.

Theorem 2.1. According to the deﬁnition of Φˆn(Θˆn)(see Eq.(1)) we immediately have

min

RpΦp(Θp)≤min

RqΦq(Θq)∀q < p.

The new algorithm we discuss in this paper proposes to deﬁne a sequence (ˆxk

ˆn)kof vectors

that converges to x?unique minimizer of the quadratic form q. The update formulae reads:

˜xk+1

n= ˜xk

n+θk

n∇˜xnq(ˆxk

ˆn),

where we recall that ˆnis an arbitrarily chosen integer. Then ˆxk+1

ˆn=Lˆn

n=1 ˜xk+1

n.

We shall explain now how one can accurately computes the vector step-length Θk

ˆnat each

iteration k. It is assumed that qis a quadratic form. From Eq.(1) using the chain rule, we obtain

4 MULTI-STEEPEST DESCENT ALGORITHM

the Jacobian vector Φ0

ˆn(Θˆn)∈Rˆngiven by

(2) (Φ0

ˆn(Θˆn))j=∇˜xjq(ˆxk

ˆn)T∇˜xjq ˆxk

ˆn+

ˆn

M

n=1

θn∇˜xnq(ˆxk

ˆn)!∈R,

and the Hessian matrix Φ00

ˆn(Θˆn)∈Rˆn×ˆnis given by

(Φ00

ˆn(Θˆn))i,j =∇˜xjq(ˆxk

ˆn)T∇˜xi∇˜xjq ˆxk

ˆn+

ˆn

M

n=1

θn∇˜xnq(ˆxk

ˆn)!∇˜xjq(ˆxk

ˆn).

It is worth noticing that the matrix ∇˜xi∇˜xjqˆxk

ˆn+Lˆn

n=1 θn∇˜xnq(ˆxk

ˆn)is a bloc portion of the

Hessian matrix A. However if the gradient ∇˜xnq∈Rm

ˆnassumes an extension by zero (denoted

by e

∇˜xiq) to Rmso the matrix Φ00

ˆn(Θˆn) has therefore the simplest implementable form

(3) (Φ00

ˆn(Θˆn))i,j = (e

∇˜xjq(ˆxk

ˆn))TAe

∇˜xjq(ˆxk

ˆn).

We thus have the expansion Φˆn(Θk

ˆn) = Φˆn(0) + (Θk

ˆn)TΦˆn(0) + 1

2(Θk

ˆn)TΦ00

ˆn(0)Θk

ˆn, with 0:=

(0, .., 0)T∈Rˆn. Then the vector Θk

ˆnthat annuls the gradient writes:

(4) Θk

ˆn=−Φ00

ˆn(0)−1Φ0

ˆn(0).

Algorithm 1has therefore a bloc structure which can be solved in parallel. This is due to the fact

that partial derivatives can be computed independently. The new algorithm is thus as follows

(see Algorithm 2)

Algorithm 2: Enhanced steepest descent.

k= 0;

Input: ˆx0

ˆn∈Rm;

1while Convergence do

2forall the 1≤n≤ˆndo

3˜xk

n=πn(ˆxk

ˆn);

4rn=∇˜xk

nq(ˆxk

ˆn);

5resize(rn) (i.e. extension by zero means simply project on Rm);

6end

7Assemble Φ0

ˆn(0) with element (Φ0

ˆn(0))j=rT

jrjaccording to Eq.(2);

8Assemble Φ00

ˆn(0) with element (Φ00

ˆn(0))i,j =rT

iArjaccording to Eq.(3);

9Compute Θk

ˆnsolution of Eq.(4);

10 Update ˆxk+1

ˆn= ˆxk

ˆn+Lnθn∇˜xnq(ˆxk

ˆn);

11 k=k+ 1;

12 end

3. Application to a parabolic optimal control problem

In this part we are interested in the application of Algorithm 2in a ﬁnite element computa-

tional engineering problem involving optimization with constrained PDE. In particular, we deal

with the optimal control problem of a system, which is governed by the heat equation. We shall

present two types of control problems. The ﬁrst concerns the distributed optimal control and

MULTI-STEEPEST DESCENT ALGORITHM 5

the second concerns the Dirichlet boundary control. The main diﬀerence from the algorithm just

presented in linear algebra is that the decomposition is applied on the time domain when the con-

trol. This technique is not classical, we may refer to a similar approaches that has been proposed

for the time domain decomposition in application to the control problem, for instance [19,17,18]

which basically they use a variant of the parareal in time algorithm [15].

3.1. Distributed optimal control problem. Let us brieﬂy present the steepest descent method

applied to the following optimal control problem: ﬁnd v?such that

(5) J(v?) = min

v∈L2(0,T ;L2(Ωc)) J(v),

where Jis a quadratic cost functional deﬁned by

(6) J(v) = 1

2ky(T)−ytarget k2

2+α

2ZI

kvk2

cdt,

where ytarget is a given target state and y(T) is the state variable at time T > 0 of the heat

equation controlled by the variable vover I:= [0, T ]. The Tikhonov regularization parameter α

is introduced to penalize the control’s L2-norm over the time interval I. The optimality system

of our problem reads: ∂ty−σ∆y=Bv, on I×Ω,

y(t= 0) = y0.

(7)

∂tp+σ∆p= 0,on I×Ω,

p(t=T) = y(T)−ytarget .

(8)

∇J(v) = αv +BTp= 0,on I×Ω.(9)

In the above equations, the operator Bis a linear operator that distributes the control in Ωc,

obviously Bstands for the indicator of Ωc⊂Ω, the state variable pstands for the Lagrange

multiplier (adjoint state) solution of the backward heat equation Eq.(8), Eq.(7) is called the

forward heat equation.

3.2. Dirichlet boundary optimal control problem. In this subsection we are concerned with

the PDE constrained Dirichlet boundary optimal control problem, where we aim at minimizing

the cost functional JΓdeﬁned by

(10) JΓ(vΓ) = 1

2kyΓ(T)−ytarget k2

2+α

2ZI

kvΓk2

Γdt,

where the control variable vΓis only acting on the boundary Γ ⊂∂Ω. Here too, ytarget is a given

target state (not necessary equal the one deﬁned in the last subsection ! ) and yΓ(T) is the state

variable at time T > 0 of the heat equation controlled by the variable vΓduring the time interval

I:= [0, T ]. As before αis a regularization term. The involved optimality system reads

∂tyΓ−σ∆yΓ=fon I×Ω

yΓ=vΓon I×Γ

yΓ=gon I× {∂Ω\Γ}

yΓ(0) = y0

(11)

∂tpΓ+σ∆pΓ= 0 on I×Ω

pΓ= 0 on I×∂Ω

pΓ(T) = yΓ(T)−ytarg et

(12)

∇JΓ(vΓ) = αvΓ−(∇pΓ)T~n = 0 on I×Γ,(13)

where f∈L2(Ω) is any source term, g∈L2(Γ) and ~n is the outward unit normal on Γ. the

state variable pΓstands for the Lagrange multiplier (adjoint state) solution of the backward heat

6 MULTI-STEEPEST DESCENT ALGORITHM

equation Eq.(12). Both functions fand gwill be given explicitly for each numerical test that

we consider in the numerical experiment section.

3.3. Steepest descent algorithm for optimal control of constrained PDE. In the optimal

control problem, the evaluation of the gradient as it is clear in Eq.(9) (respectively (13)) requires

the evaluation of the time dependent Lagrange multiplier p(respectively pΓ). This fact, makes the

steepest descent optimization algorithm slightly diﬀers from the Algorithm 1already presented.

Let us denote by kthe current iteration superscript. We suppose that v0is known. The ﬁrst

order steepest descent algorithm updates the control variable as follows:

(14) vk=vk−1+θk−1∇J(vk−1),for k≥1,for the distributed control

respectively as

(15) vk

Γ=vk−1

Γ+θk−1

Γ∇JΓ(vk−1

Γ),for k≥1,for the Dirichlet control

The step-length θk−1∈R−\{0}in the direction of the gradient ∇J(vk−1) = αvk−1+BTpk−1

(respectively ∇JΓ(vk−1

Γ) = αvΓ−(∇pΓ)T~n) is computed as :

θk−1=−k∇J(vk−1)k2

c,I /k∇J(vk−1)k2

∇2Jfor the distributed control.

respectively as

θk−1

Γ=−k∇JΓ(vk−1

Γ)k2

c,I /k∇JΓ(vk−1

Γ)k2

∇2JΓfor the Dirichlet control.

The above step-length θk−1(respectively θk−1

Γ) is optimal (see e.g. [8]) in the sense that it

minimizes the functional θ→J(vk−1+θ∇J(vk−1)) (respectively θ→JΓ(vk−1

Γ+θ∇JΓ(vk−1

Γ))).

The rate of convergence of this technique is κ−1

κ+1 2, where κis the condition number of the

quadratic form, namely the Hessian of the cost functional J(respectively JΓ).

3.4. Time-domain decomposition algorithm. Consider ˆnsubdivisions of the time interval

I=∪ˆn

n=1In, consider also the following convex cost functional J:

J(v1, v2, .., vˆn) = 1

2kY(T)−ytarget k2

2+α

2

ˆn

X

n=1 ZIn

kvnk2

cdt,(16)

JΓ(v1,Γ, v2,Γ, .., vˆn,Γ) = 1

2kYΓ(T)−ytarget k2

2+α

2

ˆn

X

n=1 ZIn

kvnk2

Γdt,(17)

where vn, n = 1, ..., ˆnare control variables with time support included in In, n = 1, ..., ˆn. The

state Y(T) (respectively YGamma) stands for the sum of state variables Yn(respectively Yn,Γ)

which are time-dependent state variable solution to the heat equation controlled by the variable

vn(respectively vn,Γ). . Obviously because the control is linear the state Ydepends on the

concatenation of controls v1, v2, .., vˆnnamely v=Pn= ˆn

n=1 vn.

Let us deﬁne Θˆn:= (θ1, θ2, ..., θ ˆn)Twhere θn∈R−\{0}. For any admissible control w=

Pˆn

nwn, we also deﬁne ϕˆn(Θˆn) := J(v+Pˆn

n=1 θnwn), which is quadratic. We have:

(18) ϕˆn(Θˆn) = ϕˆn(0)+ΘT

ˆn∇ϕˆn(0) + 1

2ΘT

ˆn∇2ϕˆn(0)Θˆn,

where 0= (0, ..., 0)T. Therefore we can write ∇ϕˆn(Θˆn)∈Rˆnas ∇ϕˆn(Θˆn) = D(v , w) +

H(v, w)Θˆn, where the Jacobian vector and the Hessian matrix are given respectively by:

D(v, w) := (h∇J(v),π1(w)ic,...,h∇J(v),πˆn(w)ic)T∈Rˆn,

H(v, w) := (Hn,m )n,m,for Hn,m =hπn(w),πm(w)i∇2J.

MULTI-STEEPEST DESCENT ALGORITHM 7

Here, (πn) is the restriction over the time interval In, indeed πn(w) has support on Inand

assumes extension by zero in I. The solution Θ?

ˆnof ∇ϕˆn(Θˆn) = 0can be written in the form:

(19) Θ?

ˆn=−H−1(v, w)D(v, w).

In the parallel distributed control problem, we are concerned with the following optimality sys-

tem: ∂tYn−σ∆Yn=Bvn,on I×Ω,

Yn(t= 0) = δ0

ny0.

(20)

Y(T) =

ˆn

X

n=1

Yn(T)(21)

∂tP+σ∆P= 0,on I×Ω,

P(t=T) = Y(T)−ytarget .

(22)

∇J(

ˆn

X

n=1

vn) = BTP+α

ˆn

X

n=1

vn= 0,on I×Ω.(23)

where δ0

nstands for the function taking value ”1” only if n= 0, else it takes the value ”0”. The

Dirichlet control problem we are concerned with:

∂tYn,Γ−σ∆Yn,Γ=fon I×Ω

Yn,Γ=vn,Γon I×Γ

Yn,Γ=gon I× {∂Ω\Γ}

Yn,Γ(0) = δ0

ny0.

(24)

YΓ(T) =

ˆn

X

n=1

Yn,Γ(T)(25)

∂tPΓ+σ∆PΓ= 0 on I×Ω

PΓ= 0 on I×∂Ω

PΓ(T) = YΓ(T)−ytarget.

(26)

∇JΓ(

ˆn

X

n=1

vn,Γ) = −∇PΓT~n +α

ˆn

X

n=1

vn,Γ= 0 on I×Γ.(27)

The resolution of Eqs. (20) and (24) with respect to nare fully performed in parallel over the

time interval I. It is recalled that the superscript kdenotes the iteration index. The update

formulae for the control variable vkis given by:

vk

n=vk−1

n+θk−1

nBTPk−1+α

ˆn

X

n=1

vk−1

n.

respectively as

vk

n,Γ=vk−1

n,Γ+θk−1

n,Γ−∇Pk−1

ΓT~n +α

ˆn

X

n=1

vk−1

n,Γ.

We shall drop in the following the index Γof the cost functional J. This index

would be only used to specify which cost function is in consideration. unless the

driven formulation apply for distributed as well as boundary control.

We show hereafter how to assemble vector step-length Θk

ˆnat each iteration. For the purposes

of notation we denote by Hkthe k-th iteration of the Hessian matrix H(∇J(vk),∇J(vk)) and by

Dkthe k-th iteration of the Jacobian vector D(∇J(vk),∇J(vk)). The line-search is performed

8 MULTI-STEEPEST DESCENT ALGORITHM

with quasi-Newton techniques that uses at each iteration ka Hessian matrix Hkand Jacobian

vector Dkdeﬁned respectively by:

Dk:= h∇J(vk),π1∇J(vk)ic, .., h∇J(vk),πˆn∇J(vk)icT,(28)

(Hk)n,m := hπn∇J(vk),πm∇J(vk)i∇2J.(29)

The spectral condition number of the Hessian matrix ∇2Jis denoted as: κ=κ(∇2J) :=

λmaxλ−1

min, with λmax := λmax(∇2J) the largest eigenvalue of ∇2Jand λmin := λmin(∇2J)

its smallest eigenvalue.

According to Eq.(19) we have

(30) Θk

ˆn=−H−1

kDk.

From Eq.(18) we have:

(31) J(vk+1) = J(vk) + (Θk

ˆn)TDk+1

2(Θk

ˆn)THkΘk

ˆn.

Our parallel algorithm to minimize the cost functional Eq.(16) and (17), is stated as follows (see

Algorithm 3).

Algorithm 3: Enhanced steepest descent algorithm for the optimal control prob-

lem.

0Input:v0

1while Convergence do

2forall the 1≥n≥ˆndo

3Solve Yn(T)(vk

n) of Eq.(20)(respectively Eq.(24)) in parallel for all 1 ≤n≤ˆn;

4end

5Compute P(t) with the backward problem according to Eq.(22) (respectively Eq.(26)) ;

6forall the 1≥n≥ˆndo

7Compute (Dk)nof Eq.(28)in parallel for all 1 ≤n≤ˆn;

8end

9Gather (Dk)nfrom processor n, 2 ≤n≤ˆnto master processor;

10 Assemble the Hessian matrix Hkaccording to Eq.(29) with master processor;

11 Compute the inversion of Hkand calculate Θk

ˆnusing Eq.(30);

12 Broadcast θk

nfrom master processor to all slaves processors;

13 Update time-window-control variable vk+1

nin parallel as :

vk+1

n=vk

n+θk

nπn∇J(vk)for all 1 ≤n≤ˆn,

and go to step 2;

14 k=k+ 1;

15 end

Since (vn)nhas disjoint time-support, thanks to the linearity, the notation en∇J(vk)is

nothing but ∇J(vk

n), where vkis the concatenation of vk

1, . . . , vk

ˆn. In Algorithm 3steps 9,10,11,

12 and 13 are trivial tasks in regards to computational eﬀort.

MULTI-STEEPEST DESCENT ALGORITHM 9

4. Convergence analysis of Algorithm 3

This section provides the proof of convergence of Algorithm 3. In the sequel, we suppose that

k∇J(vk)kcdoes not vanish; otherwise the algorithm has already converged.

proposition 4.1. The increase in value of the cost functional Jbetween two successive controls

vkand vk+1 is bounded below by:

(32) J(vk)−J(vk+1)≥1

2κ(Hk)

k∇J(vk)k4

c

k∇J(vk)k2

∇2J

.

Proof. Using Eq.(30) and Eq.(31), we can write:

(33) J(vk)−J(vk+1) = 1

2DT

kH−1

kDk.

Preleminaries: From the deﬁnition of the Jacobian vector Dkwe have

kDkk2

2=

ˆn

X

n=1

h∇J(vk),πn(∇J(vk))i2

c,

=

ˆn

X

n=1

hπn(∇J(vk)),πn(∇J(vk))i2

c,

=

ˆn

X

n=1

ken(∇J(vk))k4

c,

=k∇J(vk)k4

c.

Furthermore since Hkis an SPD matrix we have λmin(H−1

k) = 1

λmax(Hk),from which we deduce:

1

λmin(Hk)≥1

1

ˆn1T

ˆnHk1ˆn.Moreover, we have:

DT

kH−1

kDk=DT

kH−1

kDk

kDkk2

2

kDkk2

2≥λmin(H−1

k)kDkk2

2

=λmin(H−1

k)λmin(Hk)k∇J(vk)k4

c

λmin(Hk)

≥λmin(Hk)

λmax(Hk)

k∇J(vk)k4

c

1

ˆn1T

ˆnHk1ˆn

=ˆn

κ(Hk)k∇J(vk)k−2

∇2Jk∇J(vk)k4

c.

Since the partition number ˆnis greater than or equal to 1, we conclude that :

(34) DT

kH−1

kDk≥k∇J(vk)k−2

∇2Jk∇J(vk)k4

c

κ(Hk).

Hence, using Eq.(33) we get the stated result.

Theorem 4.2. For any partition ˆnof sub intervals, the control sequence (vk)k≥1of Algorithm 3

converges to the optimal control vkunique minimizer of the quadratic functional J. Furthermore

we have:

kvk−v?k2

∇2J≤rkkv0−v?k2

∇2J,

where the rate of convergence r:= 1−4κ

κ(Hk)(κ+1)2satisﬁes 0≤r < 1.

10 MULTI-STEEPEST DESCENT ALGORITHM

Proof. We denote by v?the optimal control that minimizes J. The equality

J(v) = J(v?) + 1

2hv−v?, v −v?i∇2J=J(v?) + 1

2kv−v?k2

∇2J,

holds for any control v; in particular we have:

J(vk+1) = J(v?) + 1

2kvk+1 −v?k2

∇2J,

J(vk) = J(v?) + 1

2kvk−v?k2

∇2J.

Consequently, by subtracting the equations above, we obtain

(35) J(vk+1)−J(vk) = 1

2kvk+1 −v?k2

∇2J−1

2kvk−v?k2

∇2J.

Since Jis quadratic, we have ∇2J(vk−v?) = ∇J(vk), that is vk−v?= (∇2J)−1∇J(vk).

Therefore we deduce:

kvk−v?k2

∇2J=hvk−v?, vk−v?i∇2J

(36)

=hvk−v?,∇2J, vk−v?ic

=h(∇2J)−1∇J(vk),∇2J, (∇2J)−1∇J(vk)ic

=h∇J(vk),(∇2J)−1,∇J(vk)ic

=k∇J(vk)k2

(∇2J)−1.

Because of Eq.(33), we also have

J(vk+1)−J(vk) = −1

2DT

kH−1

kDk.

Using Eq.(35) and the above, we ﬁnd that:

kvk+1 −v?k2

∇2J=kvk−v?k2

∇2J−DT

kH−T

kDk.

Moreover, according to Eqs (34)-(36), we obtain the following upper bound:

kvk+1 −v?k2

∇2J≤ kvk−v?k2

∇2J−1

κ(Hk)

k∇J(vk)k4

c

k∇J(vk)k2

∇2J

≤ kvk−v?k2

∇2J1−1

κ(Hk)

k∇J(vk)k4

c

k∇J(vk)k2

∇2Jk∇J(vk)k2

(∇2J)−1.(37)

Using the Kantorovich inequality [14,1] (see also The Appendix) :

(38) k∇J(vk)k4

c

k∇J(vk)k2

∇2Jk∇J(vk)k2

(∇2J)−1

≥4λmaxλmin

(λmax +λmin)2.

Then

1−1

κ(Hk)

k∇J(vk)k4

c

k∇J(vk)k2

∇2Jk∇J(vk)k2

(∇2J)−1

≤1−4κ

κ(Hk)(κ+ 1)2.

Finally we obtain the desired results for any partition to ˆnsubdivision, namely

kvk−v?k2

∇2J≤1−4κ

κ(Hk)(κ+ 1)2kkv0−v?k2

∇2J.

The proof is therefore complete.

Remark 4.1. Remark that the proof stands correct for the boundary control, need just to change

the subscript ”c” indicating the distributed control region Ωc, replace it by ”Γ” to indicate the

boundary control on Γ⊂∂Ω.

MULTI-STEEPEST DESCENT ALGORITHM 11

Remark 4.2. Remark that for ˆn= 1, we immediately get the condition number κ(Hk) = 1 and

we recognize the serial steepest gradient method, which has convergence rate κ−1

κ+1 2.

It is diﬃcult to pre-estimate the spectral condition number κ(Hk)(ˆn) (is a function of ˆn)

that play an important role and contribute to the evaluation of the rate of convergence as

our theoretical rate of convergence stated. We present in what follows numerical results that

demonstrate the eﬃciency of our algorithm, Tests consider examples of well-posed and ill-posed

control problem.

5. Numerical experiments

We shall present the numerical validation of our method in tow stages. In the ﬁrst stage, we

consider a linear algebra framework where we construct a random matrix-based quadratic cost

function that we minimize using Algorithm 2. In the second stage, we consider the two optimal

control problems presented in sections 3.1 and in 3.2 for the distributed- and Dirchlet boundary-

control respectively. In both cases we minimize a quadratic cost function properly deﬁned for

each handled control problem.

5.1. Linear algebra program. This subsection treat basically the implementation of Algo-

rithm 2. The program was implemented using the scientiﬁc programming language Scilab [26].

We consider the minimization of a quadratic form qwhere the matrix Ais an SPD m-by-mma-

trix and a real vector b∈Rm∩rank(A) are generated by hand (see below for their constructions).

We aim at solving iteratively the linear system Ax =b, by minimizing

(39) q(x) = 1

2xTAx −xTb.

Let us denote by ˆnthe partition number of the unknown x∈Rm. The partition is supposed

to be uniform and we assume that ˆndivides mwith a null rest.

We give in Table 1aScilab function that builds the vector step-length Θk

ˆnas stated in

Eq. (4). In the practice we randomly generate an SPD sparse matrix A= (α+γm)IRm+R,

where 0 < α < 1, γ > 1, IRmis the m-by-midentity matrix and Ris a symmetric m-by-m

random matrix. This way the matrix Ais symmetric and diagonally dominant, hence SPD.

It is worthy noticing that the role of αis regularizing when rapidly vanishing eigenvalues of A

are generated randomly. This technique helps us to manipulate the coercivity of the handled

problem hence its spectral condition number.

For such matrix Awe proceed to minimize the quadratic form deﬁned in Eq.(39) with several

ˆn-subdivisions.

The improvement quality of the algorithm against the serial case ˆn= 1 in term of iteration

number is presented in Figure. 1. In fact, the left hand side of Figure. 1presents the cost function

minimization versus the iteration number of the algorithm where several choices of partition on

ˆnare carried out. In the right hand side of the Figure. 1we give the logarithmic representation

of the relative error kxk−x?k2

kx?k2, where x?is the exact solution of the linear system at hand.

5.2. Heat optimal control program. We discuss in this subsection the implementation results

of Algorithm 3for the optimization problems presented in section 3. Our tests deal with the 2D-

heat equation on the bounded domain Ω = [0,1]×[0,1]. We consider, three types of test problems

in both cases of distributed and Dirichlet controls. Tests vary according to the theoretical

diﬃculty of the control problem [6,3,10]. Indeed, we vary the regularization parameter αand

also change the initial and target solutions in order to handle more severe control problems as

has been tested for instance in [6].

Numerical tests concern the minimization of the quadratic cost functionals J(v) and JΓ(vΓ)

using Algorithm 3. It is well known that in the case αvanishes the control problem becomes

12 MULTI-STEEPEST DESCENT ALGORITHM

1f u n c t i o n [ P] = Bu i l d Hk ( n , A, b , x k , d Jk )

2m= s i z e (A , 1 ) ; l=m/n ; i i =modulo( m, n ) ;

3i f i i ˜=0 then

4printf(” P l e a s e c h o s e an o t h e r n ! ” ) ;

5abort ;

6end

7dJkn=z e r o s (m, n ) ; Dk= [ ] ;

8f o r i = 1:n

9dJ kn ( ( i −1)∗l + 1: i ∗l , i )= dJk ( ( i −1) ∗l +1: i ∗l ) ;

10 Dk( i )=dJ kn ( : , i ) ’ ∗(A∗xk−b) ;

11 end

12 Hk = [ , ] ;

13 f o r i = 1:n

14 f o r j =i : n

15 Hktmp=A∗d Jkn ( : , j ) ;

16 Hk( i , j )=dJk n ( : , i ) ’ ∗Hktmp;

17 Hk( j , i )=Hk( i , j ) ;

18 end

19 end

20 theta=−Hk\Dk ;

21 P = eye (m,m) ;

22 f o r i = 1:n

23 P( ( i −1)∗l + 1: i ∗l , ( i −1)∗l +1 : i ∗l )=t h e t a ( i ) . ∗eye( l , l ) ;

24 end

25 endfunction

.

Table 1. Scilab function to build the vector step length, for the linear algebra program.

Figure 1. Performance in term of iteration number: Several decomposition on

ˆn. Results from the linear algebra Scilab program.

an ”approximated” controllability problem. Therefore the control variable tries to produce a

solution that reaches as close as it ”can” the target solution. With this strategy, we accentuate

the ill-conditioned degree of the handled problem. We also consider an improper-posed problems

for the controllability approximation, where the target solution doesn’t belong to the space of

the reachable solutions. No solution exists thus for the optimization problem i.e. no control

exists that enables reaching the given target !

MULTI-STEEPEST DESCENT ALGORITHM 13

For reading conveniences and in order to emphasize the role of the parameter αon numerical

tests, we tag problems that we shall consider as Pα

iwhere the index irefers to the problem

among {1,2,3,4}. The table below resumes all numerical test that we shall experiences

– Minimize J(v) distributed control Minimize JΓ(vΓ) boundary control

Moderate α= 1 ×10−02 well-posed problem ill-posed problem

corresponding data in (Pα

1), (Pα

2) corresponding data in (Pα

3)

Vanishing α= 1 ×10−08 ill-posed problem sever ill-posed problem

corresponding data in (Pα

1),(Pα

2) corresponding data in (Pα

3)

Solution does not exist sever ill-posed problem sever ill-posed problem

corresponding data in (Pα

4) corresponding data in (Pα

4)

We suppose from now on that the computational domain Ω is a polygonal domain of the

plane R2. We then introduce a triangulation Thof Ω; the subscript hstands for the largest

length of the edges of the tringles that constitute Th. The solution of the heat equation at a

given time tbelongs to H1(Ω). The source terms and other variables are elements of L2(Ω).

Those inﬁnite dimensional spaces are therefore approximated with the ﬁnite-dimensional space

Vh, characterized by P1the space of the polynomials of degree ≤1 in two variables (x1, x2).

We have Vh:= {uh|uh∈C0(Ω), uh|K∈P1,for all K∈ Th}. In addition, Dirichlet boundary

conditions (where the solution is in H1

0(Ω) i.e. vanishing on boundary ∂Ω) are taken into account

via penalization of the vertices on the boundaries. The time dependence of the solution is

approximated via the implicit Euler scheme. The inversion operations of matrices is performed

by the umfpak solver. We use the trapezoidal method in order to approximate integrals deﬁned

on the time interval.

The numerical experiments were run using a parallel machine with 24 CPU’s AMD with 800

MHz in a Linux environment. We code two FreeFem++ [22] scripts for the distributed and

Dirichlet control. We use MPI library in order to achieve parallelism.

Tests that concern the distributed control problem are produced with control that acts on

Ωc⊂Ω, with Ωc= [0,1

3]×[0,1

3], whereas Dirichlet boundary control problem, the control acts

on Γ ⊂∂Ω, with Γ = {(x1, x2)∈∂Ω,|x2= 0}. The time horizon of the problem is ﬁxed to

T= 6.4 and the small time step is τ= 0.01. In order to have a better control of the time

evolution we put the diﬀusion coeﬃcient σ= 0.01.

5.2.1. First test problem: Moderate Tikhonov regularization parameter α.We consider an opti-

mal control problem on the heat equation. The control is considered ﬁrst to be distributed and

then Dirichlet. For the distributed optimal control problem we ﬁrst use the functions

y0(x1, x2) = exp −γ2π(x1−.7)2+ (x2−.7)2

ytarget (x1, x2) = exp −γ2π(x1−.3)2+ (x2−.3)2,

(Pα

1)

as initial condition and target solution respectively. The real valued γis introduced to force the

Gaussian to have support strictly included in the domain and verify the boundary conditions.

The aim is to minimize the cost functional deﬁned in Eq. (6). The decay of the cost function

with respect to the iterations of our algorithm is presented in Figure. 2on the left side, and

the same results are given with respect to the computational CPU’s time (in sec) on the right

side. We show that the algorithm accelerates with respect to the partition number ˆnand also

preserves the accuracy of the resolution. Indeed, all tests independently of ˆnalways converge to

the unique solution. This is in agreement with Theorem (4.2), which proves the convergence of

the algorithm to the optimal control (unique if it exists [16]) for an arbitrary partition choice ˆn.

14 MULTI-STEEPEST DESCENT ALGORITHM

Figure 2. First test problem, for Pα

1: Normalized and shifted cost functional

values versus iteration number (left) and versus computational time (right) for

several values of ˆn(i.e. the number of processors used).

Figure 3. Snapshots in ˆn= 1,16 of the distributed optimal control on the left

columns and its corresponding controlled ﬁnal state at time T: y(T) on the right

columns. The test case corresponds to the control problem Pα

1, where αis taken

as α= 1 ×10−02 . Same result apply for diﬀerent choice of ˆn.

MULTI-STEEPEST DESCENT ALGORITHM 15

We test a second problem with an a priori known solution of the heat equation. The considered

problem has

y0(x1, x2) = sin(πx1) sin(πx2)

ytarget (x1, x2) = exp(−2π2σT ) sin(πx1) sin(πx2),

(Pα

2)

as initial condition and target solution respectively. Remark that the target solution is taken as

a solution of the heat equation at time T. The results of this test are presented in Figure. 4,

which shows the decay in values of the cost functional versus the iterations of the algorithm on

the left side and versus the computational CPU’s time (in sec) on the right side.

Figure 4. First test problem, for Pα

2: Normalized cost functional values versus

computational CPU time for several values of ˆn(i.e. the number of processors

used).

We give in Figure. 3and Figure. 5several rows value snapshots (varying the ˆn) of the control

and its corresponding controlled ﬁnal solution y(T). Notice the stability and the accuracy of the

method with any choice of ˆn. In particular the shape of the resulting optimal control is unique

as well as the controlled solution y(T) doesn’t depend on ˆn.

For the Dirichlet boundary control problem we choose the following functions as source term,

initial condition and target solution:

f(x1, x2, t)=3π3σexp(2π2σt)(sin(πx1) + sin(πx2))

y0(x1, x2) = π(sin(πx1) + sin(πx2))

ytarget (x1, x2) = πexp(2π2σ)(sin(πx1) + sin(πx2)),

(Pα

3)

respectively. Because of the ill-posed character of this problem, its optimization leads to results

with hight contrast in scale. We therefore preferred to summarize the optimizations results in

Table 3instead of Figures.

Remark 5.1. Because of the linearity and the superposition property of the heat equation, it can

be shown that problems (Pα

2and Pα

3) mentioned above are equivalent to a control problem which

has null target solution.

16 MULTI-STEEPEST DESCENT ALGORITHM

Figure 5. Snapshots in ˆn= 1,16 of the distributed optimal control on the

left columns and its corresponding controlled ﬁnal state at time T: y(T) on the

right columns. The test case corresponds to the control problem Pα

2, where

α= 1 ×10−02 . Same results apply for diﬀerent choice of ˆn.

Test problem Results

Pα

1α= 1 ×10−02

Quantity ˆn= 1 ˆn= 2 ˆn= 4 ˆn= 8 ˆn= 16

Number of iterations k100 68 63 49 27

walltime in sec 15311.6 15352.3 14308.7 10998.2 6354.56

kYk(T)−ytarg etk2/kytarg etk20.472113 0.472117 0.472111 0.472104 0.472102

R(0,T )kvkk2

cdt 0.0151685 0.0151509 0.0151727 0.0152016 0.015214

Pα

2α= 1 ×10−02

Quantity ˆn= 1 ˆn= 2 ˆn= 4 ˆn= 8 ˆn= 16

Number of iterations k60 50 45 40 35

walltime in sec 3855.21 3726.28 4220.92 3778.13 3222.78

kYk(T)−ytarg etk2/kytarg etk28.26 ×10−08 8.26 ×10−08 8.15 ×10−08 8.15 ×10−08 8.14 ×10−08

R(0,T )kvkk2

cdt 1.68 ×10−07 1.68 ×10−07 1.72 ×10−07 1.72 ×10−07 1.72 ×10−07

Pα

2α= 1 ×10−08

Quantity ˆn= 1 ˆn= 2 ˆn= 4 ˆn= 8 ˆn= 16

Number of iterations k60 50 40 30 20

walltime in sec 3846.23 4654.34 3759.98 2835.31 1948.4

kYk(T)−ytarg etk2/kytarg etk23.93 ×10−08 1.14 ×10−08 5.87 ×10−09 2.04 ×10−09 1.76 ×10−09

R(0,T )kvkk2

cdt 5.42 ×10−07 4.13 ×10−06 2.97 ×10−04 3.64 ×10−03 2.51 ×10−03

Table 2. Results’ summary of Algorithm 3applied on the distributed control

problems Pα

1and Pα

2.

5.2.2. Second test problem: vanishing Tikhonov regularization parameter α.In this section, we

are concerned with the ”approximate” controllability of the heat equation, where the regulariza-

tion parameter αvanishes, practically we take α= 1 ×10−08. In this case, problems Pα

2and Pα

3,

in the continuous setting are supposed to be well posed (see for instances [9,21]). However, may

not be the case in the discretized settings; we refer for instance to [10] (and reference therein)

for more details.

MULTI-STEEPEST DESCENT ALGORITHM 17

Figure 6. Several rows value snapshots in ˆnof the Dirichlet optimal control on

the left columns and its corresponding controlled ﬁnal state at time T: y(T) on

the right columns. The test case corresponds to the control problem Pα

3, where

α= 1 ×10−02 .

18 MULTI-STEEPEST DESCENT ALGORITHM

Figure 7. Normalized and shifted cost functional values versus computational

CPU time for several values of ˆn(i.e. the number of processors used), Distributed

control problem Pα

2whith α= 1 ×10−08 .

Test problem Results

Pα

3α= 1 ×10−02

Quantity ˆn= 1 ˆn= 2 ˆn= 4 ˆn= 8 ˆn= 16

Number of iterations 40 40 30 18 10

walltime in sec 12453.9 12416.1 9184.28 5570.54 3158.97

kYΓ(T)−ytarget k2/kytarget k28.54 ×10+06 0.472488 0.0538509 0.0533826 0.0534024

R(0,T )kvk2

Γdt 2.79 ×10+08 1.96 ×10+07 31.4193 138.675 275.08

Pα

3α= 1 ×10−08

Quantity ˆn= 1 ˆn= 2 ˆn= 4 ˆn= 8 ˆn= 16

Number of iterations 40 40 30 27 10

walltime in sec 1248.85 1248.97 916.232 825.791 325.16

kYΓ(T)−ytarget k2/kytarget k28.85 ×10+06 0.151086 0.0292072 0.0278316 0.0267375

R(0,T )kvk2

Γdt 7.92 ×10+08 2.30 ×10+07 1.27 ×10+07 1.47 ×10+07 1.58 ×10+06

Table 3. Results’ summary of Algorithm 3applied on the Dirichlet boundary

control problem Pα

3.

Table 2contains the summarized results for the convergence of the distributed control problem.

On the one hand, we are interested in the error given by our algorithm for several choices of

partition number ˆn. On the other hand, we give the L2(0, T ;L2(Ωc)) of the control. We notice the

improvement in the quality of the algorithm in terms of both time of execution and control energy

consumption, namely the quantity R(0,T)kvkk2

cdt. In fact, for the optimal control framework (Pα

1

and Pα

2with α= 1 ×10−02 ), we see that, for a ﬁxed stopping criterion, the algorithm is faster

and consume the same energy independently of ˆn. In the approximate controllability framework

(Pα

2with α= 1 ×10−08 vanishes), we note ﬁrst that the general accuracy of the controlled

solution (see the error kYk(T)−ytarget k2/kytarget k2) is improved as α= 1 ×10−08 compered

with α= 1 ×10−02. Second, we note that the error diminishes when increasing ˆn, the energy

consumption rises however. The scalability in CPU’s time and number of iteration shows the

enhancement of our method when it is applied (i.e. for ˆn > 1).

Table 3contains the summarized results at the convergence of the Dircichlet boundary control

problem. This problem is known in the literature for its ill-posedness, where it may be singular

MULTI-STEEPEST DESCENT ALGORITHM 19

Figure 8. Several rows value snapshots in ˆnof the distributed optimal control

on the left columns and its corresponding controlled ﬁnal state at time T: (Y(T)

on the right columns. The test case corresponds to the control problem Pα

2,

where α= 1 ×10−08 .

20 MULTI-STEEPEST DESCENT ALGORITHM

Figure 9. Several rows value snapshots in ˆnof the Dirichlet optimal control on

the left columns and its corresponding controlled ﬁnal state at time T: YΓ(T) on

the right columns. The test case corresponds to the control problem Pα

3, where

α= 1 ×10−08 ..

MULTI-STEEPEST DESCENT ALGORITHM 21

in several cases see [3] and references therein. In fact, it is very sensitive to noise in the data.

We show in Table 3that for a big value of the regularization parameter αour algorithm behaves

already as the distributed optimal control for a vanishing α, in the sense that it consumes

more control energy to produce a more accurate solution with smaller execution CPU’s time.

It is worth noting that the serial case ˆn= 1 fails to reach an acceptable solution, whereas the

algorithm behaves well as ˆnrises.

We give in Figure. 6and Figure. 9several rows value snapshots (varying ˆn) of the Dirichlet

control on Γ. We present in the ﬁrst column its evolution during [0, T ] and on the second column

its corresponding controlled ﬁnal solution y(T) at time T; we scaled the plot of the z-range of

the target solution in both Figs.6and 9.

In each row one sees the control and its solution for a speciﬁc partition ˆn. The serial case

ˆn= 1 leads to a controlled solution which doesn’t have the same rank as ytarget, whereas as ˆn

rises, we improve the behavior of the algorithm.

It is worth noting that the control is generally active only around the ﬁnal horizon time T.

This is very clear in Figure. 6and Figure. 9(see the ﬁrst row i.e. case ˆn= 1). The nature

of our algorithm, which is based on time domain decomposition, obliges the control to act in

subintervals. Hence, the control acts more often and earlier in time (before T) and leads to a

better controlled solution y(T).

5.2.3. Third test problem: Sever ill-posed problem (no solution). In this test case, we consider a

severely ill-posed problem. In fact, the target solution is piecewise Lipschitz continuous, so that

it is not regular enough compared with the solution of the heat equation. This implies that in

our control problem, both the distributed and the Dirchlet boundary control has no solution.

The initial condition and the target solution are given by

y0(x1, x2) = π(sin(πx1) + sin(πx2))

ytarget (x1, x2) = min x1, x2,(1 −x1),(1 −x2),

(Pα

4)

respectively. A plots of the initial condition and the target solutions are given in Figure. 10.

Figure 10. Graph of initial and target solution for both distributed and

Dirichlet boundary control problem.

In Figures 11 and 12 we plot the controlled solution at time Tfor the distributed and Dirichlet

control problems respectively. We remark that for the distributed control problem the controlled

solution is smooth except in Ωc, where the control is able to ﬁt with the target solution.

22 MULTI-STEEPEST DESCENT ALGORITHM

Figure 11. Several snapshots in ˆnof ﬁnal state at time T: Y(T). The test case

corresponds to Distributed control sever Ill-posed problem Pα

4.

Test problem Results

Distributed control

Pα

4α= 1 ×10−08

Quantity ˆn= 1 ˆn= 2 ˆn= 4 ˆn= 8 ˆn= 16

Number of iterations 100 68 60 50 40

walltime in sec 6381.43 6303.67 5548.16 4676.83 3785.97

kY(T)−ytarg etk2/kytarg etk28.16 ×10−03 5.3×10−03 4.74 ×10−03 3.95 ×10−03 3.76 ×10−03

R(0,T )kvk2

cdt 0.34 3.01 52.87 52.77 2660.87

Dirichlet control

Pα

4α= 1 ×10−08

Quantity ˆn= 1 ˆn= 2 ˆn= 4 ˆn= 8 ˆn= 16

Number of iterations 25 25 20 4 1

walltime in sec 848.58 655.40 655.40 146.19 62.87

kYΓ(T)−ytarget k2/kytarget k22.85 ×10+10 3055 39.3 0.2 0.067

R(0,T )kvk2

Γdt 6.73 ×10+08 2.17 ×10+07 141.62 17.84 26758.5

Table 4. Results’ summary of Algorithm 3applied on to both distributed and

Dirichlet boundary control for the third test problem Pα

4.

MULTI-STEEPEST DESCENT ALGORITHM 23

Figure 12. Several snapshots in ˆnof ﬁnal state at time T: YΓ(T). The test

case corresponds to Dirichlet control sever Ill-posed problem Pα

4.

Remark 5.2. Out of curiosity, we tested the case where the control is distributed on the whole

domain. We see that the control succeeds to ﬁt the controlled solution to the target even if it

is not C1(Ω). This is impressive and shows the impact on the results of the regions where the

control is distributed.

We note the stability of the method of the distributed test case. However, the Dirichlet

problem test case presents hypersensitivity. In fact, in the case of ˆn= 1 the algorithm succeeds

to ﬁt an acceptable shape of the controlled solution, although still far in the scale. We note

that the time domain decomposition leads to a control which gives a good scale of the controlled

solution.

In this severely ill-posed problem, we see that some partitions may fail to produce a control

that ﬁt the controlled solution to the target. There is an exemption for the case of ˆn= 8

partitions, where we have a good reconstruction of the target. The summarized results are given

in Tables 4.

5.2.4. Regularization based on the optimal choice of partition. The next discussion concerns the

kind of situation where the partition leads to multiple solutions, which is common in ill-posed

24 MULTI-STEEPEST DESCENT ALGORITHM

problems. In fact, we discuss a regularization procedure used as an exception handling tool to

choose the best partition, giving the best solution of the handled control problem.

It is well known that ill-posed problems are very sensitive to noise, which could be present due

to numerical approximation or to physical phenomena. In that case, numerical algorithm may

blow-up and fail. We present several numerical tests for the Dirichlet boundary control, which is

a non trivial problem numerically. The results show that in general time domain decomposition

may improve the results in several cases. But scalability is not guaranteed as it is for the

distributed control. We propose a regularization procedure in order to avoid the blow-up and

also to guarantee the optimal choice of partition of the time domain. This procedure is based

on a test of the monotony of the cost function. In fact, suppose that we possess 64 processors

to run the numerical problem. Once we have assembled the Hessian Hkand the Jacobian Dk

for the partition ˆn= 64, we are actually able to get for free the results of the Hessian and the

Jacobian for all partitions ˆnthat divide 64. Hence, we can use the quadratic property of the

cost functional in order to predict and test the value of the cost function for the next iteration

without making any additional computations. The formulae is given by:

J(vk+1) = J(vk)−1

2DT

kH−1

kDk.

We present in Algorithm 4the technique that enables us to reduce in rank and compute a series

of Hessians and Jacobians for any partition ˆnthat divide the available number of processors. An

exemple of the applicability of these technique, on a 4-by-4 SPD matrix, is given in Appendix.

Algorithm 4: Reduce in rank of the partition ˆn

0Input: ˆn, Hk

ˆn, Dk

ˆn;

1n= ˆn;

2Jk+1

n/2=Jk+1

n;

3while Jk+1

n/2> Jk

ndo

4for i= 0; i≤n;i+ 2 do

5Dk

n/2i=Dk

ni+Dk

ni+1;

6for j= 0; j≤n;j+ 2 do

7Hk

n/2i,j =Hk

nj+Hk

nj+1;

8end

9end

10 Estimation of the cost Jk

n/2;

11 n=n/2;

12 end

6. Conclusion

We have presented in this article a new optimization technique to enhance the steepest descent

algorithm via domain decomposition in general and we applied our new method in particular to

time-parallelizing the simulation of an optimal heat control problem. We presented its perfor-

mance (in CPU time and number of iterations) versus the traditional steepest descent algorithm

in several and various test problems. The key idea of our method is based on a quasi-Newton

technique to perform eﬃcient real vector step-length for a set of descent directions regarding the

domain decomposition. The originality of our approach consists in enabling parallel computation

where its vector step-length achieves the optimal descent direction in a high dimensional space.

MULTI-STEEPEST DESCENT ALGORITHM 25

Convergence property of the presented method is provided. Those results are illustrated with

several numerical tests using parallel resources with MPI implementation.

Appendix A. Kantorovich matrix inequality

For the sake of completeness, we give in this appendix the Matrix Kantorovich inequality, that

justiﬁes the statement of our convergence proof. Assume that ∇2Jis symmetric positive deﬁnite

with smallest and largest eigenvalues λmin and λmax respectively. We give in the following the

matrix version of the famous Kantorovich inequality, which reads:

Theorem A.1 (see [14] for more details).Assume that Pˆn

n=1 αn= 1 where αn≥0and λn>

0∀n; we have thus :

ˆn

X

n=1

αnλn

ˆn

X

n=1

αn

λn

≤(λmax +λmin)2

4λmaxλmin

.

By diagonalizing the symmetric positive deﬁnite operator Hwe obtain: H=PΛP−1, where

Pis orthonormal operator (i.e. PT=P−1). Recall Eq.(38) that we rewrite as:

k∇J(vk)k2

∇2Jk∇J(vk)k2

(∇2J)−1

k∇J(vk)k4

c

≤(λmax +λmin)2

4λmaxλmin

.

In order to simplify the expression, we shall use dkinstead of ∇J(vk) so that the equation above

reads:

dT

k(∇2J)dkdT

k(∇2J)−1dk

(dT

kdk)2=dT

kPTΛP dk

dT

kPTP dk

dkPTΛ−1P dk

dT

kPTP dk

.

Let us deﬁne dk:= P dk, consequently the above equality becomes:

dT

kΛdk

dT

kdk

dT

kΛ−1dk

dT

kdk

=

ˆn

X

n=1

(dk)2

n

dT

kdk

λn

ˆn

X

n=1

(dk)2

n

dT

kdk

1

λn

.

We then denote by αn=(dk)2

n

dT

kdkso that Pˆn

n=1 αn= 1, and ﬁnally:

dT

kAdkdT

kA−1dk

(dT

kdk)2=

ˆn

X

n=1

αnλn

ˆn

X

n=1

αn

λn

.

Example 1. Exemple 4-by-4 SPD matrix reduced in rank using the regularization procedure

described in Algorithm 4. In order to illustrate the steps of Algorithm 4, we choose a simple

example: a matrix 4-by-4 which we are going to reduce recursively in 2-by-2 and in 1-by-1 as

follows:

6 1 2 3

1 8 2 4

2 2 12 7

3 4 7 16

7→

(6 1) (2 3)

(1 8) (2 4)

(2 2) (12 7)

(3 4) (7 16)

7→

7 5

9 6

4 19

7 23

7→ 16 11

11 42

16 11

11 42 7→ 27

53 7→ (80)

26 MULTI-STEEPEST DESCENT ALGORITHM

References

[1] Jerzy K Baksalary and Simo Puntanen. Generalized matrix versions of the cauchy-schwarz and kantorovich

inequalities. Aequationes Mathematicae, 41(1):103–110, 1991.

[2] Jonathan Barzilai and Jonathan M Borwein. Two-point step size gradient methods. IMA Journal of Numer-

ical Analysis, 8(1):141–148, 1988.

[3] Faker Ben Belgacem and Sidi Mahmoud Kaber. On the Dirichlet boundary controllability of the one-

dimensional heat equation: semi-analytical calculations and ill-posedness degree. Inverse Problems,

27(5):055012, 19, 2011.

[4] Petter Bjorstad and William Gropp. Domain decomposition: parallel multilevel methods for elliptic partial

diﬀerential equations. Cambridge University Press, 2004.

[5] Richard H. Byrd, Gabriel Lopez-Calva, and Jorge Nocedal. A line search exact penalty method using steering

rules. Math. Program., 133(1-2, Ser. A):39–73, 2012.

[6] C. Carthel, R. Glowinski, and J.-L. Lions. On exact and approximate boundary controllabilities for the heat

equation: a numerical approach. J. Optim. Theory Appl., 82(3):429–484, 1994.

[7] Augustin-Louis Cauchy. M´ethode g´en´erale pour la r´esolution des syst`emes d’´equations simultan´ees. Compte

Rendu des Scieances de L’Acad´emie des Sciences XXV, S’erie A(25):536–538, October 1847.

[8] Philippe G. Ciarlet. Introduction `a l’analyse num´erique matricielle et `a l’optimisation. Collection

Math´ematiques Appliqu´ees pour la Maˆıtrise. [Collection of Applied Mathematics for the Master’s Degree].

Masson, Paris, 1982.

[9] Jean-Michel Coron and Emmanuel Tr´elat. Global steady-state controllability of one-dimensional semilinear

heat equations. SIAM J. Control Optim., 43(2):549–569, 2004.

[10] Sylvain Ervedoza and Enrique Zuazua. The wave equation: Control and numerics. In Control of Partial

Diﬀerential Equations, Lecture Notes in Mathematics, pages 245–339. Springer Berlin Heidelberg, 2012.

[11] David J Evans. Preconditioning Methods: Theory and Applications. Gordon and Breach Science Publishers,

Inc., 1983.

[12] Luigi Grippo, Francesco Lampariello, and Stephano Lucidi. A nonmonotone line search technique for newton’s

method. SIAM Journal on Numerical Analysis, 23(4):707–716, 1986.

[13] Marcus J Grote and Thomas Huckle. Parallel preconditioning with sparse approximate inverses. SIAM Jour-

nal on Scientiﬁc Computing, 18(3):838–853, 1997.

[14] L. V. Kantorovich. Functional analysis and applied mathematics. NBS Rep. 1509. U. S. Department of

Commerce National Bureau of Standards, Los Angeles, Calif., 1952. Translated by C. D. Benster.

[15] J Lions, Yvon Maday, and Gabriel Turinici. A”parareal”in time discretization of pde’s. Comptes Rendus de

l’Academie des Sciences Series I Mathematics, 332(7):661–668, 2001.

[16] J.-L. Lions. Optimal control of systems governed by partial diﬀerential equations. Translated from the French

by S. K. Mitter. Die Grundlehren der mathematischen Wissenschaften, Band 170. Springer-Verlag, New York,

1971.

[17] Y. Maday, J. Salomon, and G. Turinici. Monotonic parareal control for quantum systems. SIAM Journal on

Numerical Analysis, 45(6):2468–2482, 2007.

[18] Yvon Maday, Mohamed-Kamel Riahi, and Julien Salomon. Parareal in time intermediate targets methods

for optimal control problems. In Kristian Bredies, Christian Clason, Karl Kunisch, and Gregory von Winckel,

editors, Control and Optimization with PDE Constraints, volume 164 of International Series of Numerical

Mathematics, pages 79–92. Springer Basel, 2013.

[19] Yvon Maday and Gabriel Turinici. A parareal in time procedure for the control of partial diﬀerential equa-

tions. C. R. Math. Acad. Sci. Paris, 335(4):387–392, 2002.

[20] Tahir Malas and Levent G¨urel. Incomplete lu preconditioning with the multilevel fast multipole algorithm

for electromagnetic scattering. SIAM Journal on Scientiﬁc Computing, 29(4):1476–1494, 2007.

[21] Sorin Micu and Enrique Zuazua. Regularity issues for the null-controllability of the linear 1-d heat equation.

Systems Control Lett., 60(6):406–413, 2011.

[22] Olivier Pironneau, Fr´ed´eric Hecht, and Jacques Morice. freefem++, www.freefem.org/, 2013.

[23] Alﬁo Quarteroni and Alberto Valli. Domain decomposition methods for partial diﬀerential equations. Nu-

merical Mathematics and Scientiﬁc Computation. The Clarendon Press, Oxford University Press, New York,

1999. Oxford Science Publications.

[24] Ulrich R¨ude. Mathematical and computational techniques for multilevel adaptive methods. SIAM, 1993.

[25] Yousef Saad. Iterative methods for sparse linear systems. Siam, 2003.

[26] Scilab Enterprises. Scilab: Le logiciel open source gratuit de calcul numrique. Scilab Enterprises, Orsay,

France, 2012.

[27] Andrea Toselli and Olof Widlund. Domain decomposition methods: algorithms and theory, volume 3.

Springer, 2005.

MULTI-STEEPEST DESCENT ALGORITHM 27

[28] Gonglin Yuan and Zengxin Wei. The Barzilai and Borwein gradient method with nonmonotone line search

for nonsmooth convex optimization problems. Math. Model. Anal., 17(2):203–216, 2012.

- A preview of this full-text is provided by Springer Nature.
- Learn more

Preview content only

Content available from Numerical Algorithms

This content is subject to copyright. Terms and conditions apply.