PreprintPDF Available

STRENGTHENING LASSERRE'S HIERARCHY IN REAL AND COMPLEX POLYNOMIAL OPTIMIZATION

Authors:
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

We establish a connection between multiplication operators and shift operators. Moreover, we derive positive semidefinite conditions of finite rank moment sequences and use these conditions to strengthen Lasserre's hierarchy for real and complex polynomial optimization. Integration of the strengthening technique with sparsity is considered. Extensive numerical experiments show that our strengthening technique can significantly improve the bound (especially for complex polynomial optimization) and allows to achieve global optimality at lower relaxation orders, thus providing substantial computational savings.
Content may be subject to copyright.
STRENGTHENING LASSERRE’S HIERARCHY IN REAL AND
COMPLEX POLYNOMIAL OPTIMIZATION
JIE WANG
Abstract. We establish a connection between multiplication operators and shift operators.
Moreover, we derive positive semidefinite conditions of finite rank moment sequences and use these
conditions to strengthen Lasserre’s hierarchy for real and complex polynomial optimization. Inte-
gration of the strengthening technique with sparsity is considered. Extensive numerical experiments
show that our strengthening technique can significantly improve the bound (especially for complex
polynomial optimization) and allows to achieve global optimality at lower relaxation orders, thus
providing substantial computational savings.
Key words. polynomial optimization, complex polynomial optimization, semidefinite relax-
ation, Lasserre’s hierarchy, multiplication operator, shift operator
MSC codes. Primary, 90C23; Secondary, 90C22,90C26
1. Introduction. Lasserre’s hierarchy [6] is a well-established scheme for glob-
ally solving (real) polynomial optimization problems and attracts a lot of attentions
of researchers from diverse fields due to its nice theoretical properties in recent years
[3, 11]. There is also a complex variant of Lasserre’s hierarchy for globally solving
complex polynomial optimization problems [5].
A bottleneck of Lasserre’s hierarchy is its limited scalability as the size of as-
sociated semidefinite relaxations grows rapidly with relaxation orders. One way for
overcoming this is exploiting structures (sparsity, symmetry) of polynomial optimiza-
tion problems to obtain structured semidefinite relaxations of reduced sizes. We refer
the reader to the recent works [12, 18, 19] on this topic. Another practical idea is
strengthening Lasserre’s hierarchy to accelerate its convergence, for instance, using
Lagrange multiplier expressions as done in [10].
In this paper we propose to strengthen Lasserre’s hierarchy using positive semidef-
inite (PSD) optimality conditions for any real and complex polynomial optimization
problem. These PSD optimality conditions arise from the characterization of nor-
mality of shift operators which is closely related to multiplication operators. Both
operators have applications in extractions of optimal solutions when solving polyno-
mial optimization problems with Lasserre’s hierarchy [4, 5]. We establish a connection
between shift operators and multiplication operators. Further, we derive PSD condi-
tions of finite rank moment sequences via shift operators. These PSD conditions are
then employed to strengthen Lasserre’s hierarchy. In particular, for real polynomial
optimization, we present an intermediate relaxation between two successive moment
relaxations; for complex polynomial optimization, we present a two-level hierarchy
of moment relaxations which thus offers one more level of flexibility. To improve
scalability, the strengthening technique is further integrated into different sparse ver-
sions of Lasserre’s hierarchy. Diverse numerical experiments are performed. It is
shown that the strengthening technique can indeed improve the bound provided by
the usual Lasserre’s hierarchy and very likely allows to achieve global optimality at
Submitted to the editors DATE.
Funding: This work was funded by National Key R&D Program of China under grant No.
2023YFA1009401, Natural Science Foundation of China under grant No. 12201618 and 12171324.
Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, China
(wangjie212@amss.ac.cn, https://wangjie212.github.io/jiewang/)
1
2JIE WANG
lower relaxation orders, especially in complex polynomial optimization.
2. Notation and preliminaries. Let Nbe the set of nonnegative integers. For
nN\ {0}, let [n]:={1,2, . . . , n}. For α= (αi)Nn, let |α|:=Pn
i=1 αi. For rN,
let Nn
r:={αNn| |α| r}and |Nn
r|stands for its cardinality. We use A0 to
indicate that the matrix Ais positive semidefinite (PSD). Let ibe the imaginary unit,
satisfying i2=1. Throughout the paper, let F {R,C}. Let F[x]:=F[x1, . . . , xn]
be the ring of multivariate polynomials in nvariables over the field F, and F[x]d
denote the subset of polynomials of degree no greater than d. A polynomial fF[x]
can be written as f=PαNnfαxαwith fαFand xα:=xα1
1· · · xαn
n. For dN,
[x]dstands for the standard monomial basis of degree up to d, and [x] stands for the
standard monomial basis.
Let adenote the conjugate of a complex number aand v(resp. A) denote
the conjugate transpose of a complex vector v(resp. a complex matrix A). We use
x= (x1, . . . , xn) to denote the conjugate of the tuple of complex variables x. We
denote by C[x,x]:=C[x1, . . . , xn, x1, . . . , xn] the complex polynomial rings in x,x. A
polynomial fC[x,x] can be written as f=P(β,γ)Nn×Nnfβ,γxβxγwith fβ,γC.
The conjugate of fis defined as f=P(β,γ)Nn×Nnfβ,γxγxβ. The polynomial f
is self-conjugate if f=f. It is clear that self-conjugate polynomials take only real
values.
2.1. The real Lasserre’s hierarchy for real polynomial optimization.
Consider the real polynomial optimization problem:
(RPOP) fmin := inf {f(x) : xK},
where fR[x] and the feasible set Kis given by
(2.1) K:={xRn:g1(x)0, . . . , gm(x)0},
for some polynomials g1, . . . , gmR[x]. By invoking Borel measures, (RPOP) admits
the following reformulation:
(2.2)
inf
µ∈M+(K)RKfdµ
s.t.RKdµ= 1,
where M+(K) denotes the set of finite positive Borel measures on K.
Suppose that y= (yα)αNnis a (pseudo-moment) sequence in R. We associate
it with a linear functional Ly:R[x]Rby
f=X
α
fαxα7− Ly(f) = X
α
fαyα.
For rN, the r-th order real moment matrix MR
r(y) is the matrix indexed by Nn
r
such that
[MR
r(y)]βγ :=Ly(xβxγ) = yβ+γ,β,γNn
r.
The real moment matrix MR(y) indexed by Nnis defined similarly. For a polynomial
g=PαgαxαR[x], the r-th order real localizing matrix MR
r(gy) associated with g
is the matrix indexed by Nn
rsuch that
[MR
r(gy)]βγ :=Ly(gxβxγ) = X
α
gαyα+β+γ,β,γNn
r.
STRENGTHENING LASSERRE’S HIERARCHY 3
The sequence yis called a real moment sequence if it can be realized by a Borel
measure µ, i.e., yα=RKxαdµfor any αNn, and yis said to be of finite rank if
µis a finitely atomic measure (that is, a linear positive combination of finitely many
Dirac measures), where the rank of yis defined as the number of atoms.
Lemma 2.1 ([8], Lemma 4.2). If yis a real moment sequence of finite rank, then
MR(y)0and the rank of yis equal to rank MR(y).
Let di:=deg(gi)/2, i = 1, . . . , m,dmin := max{⌈deg(f)/2, d1, . . . , dm}. With
rdmin, the real Lasserre’s hierarchy of moment relaxations for (RPOP) [6] is given
by
(2.3) ρr:=
inf
yLy(f)
s.t.MR
r(y)0, y0= 1,
MR
rdi(giy)0, i [m].
2.2. The complex Lasserre’s hierarchy for complex polynomial opti-
mization. Consider the complex polynomial optimization problem:
(CPOP) fmin := inf {f(x,x) : xK},
where
(2.4) K:={xCn|gi(x,x)0, i [m]},
and f, g1, . . . , gmC[x,x] are self-conjugate polynomials. By invoking Borel mea-
sures, (CPOP) also admits the following reformulation:
(2.5)
inf
µ∈M+(K)RKfdµ
s.t.RKdµ= 1,
where M+(K) denotes the set of finite positive Borel measures on K.
Suppose that y= (yβ,γ)(β,γ)Nn×Nnin Cis a (pseudo-moment) sequence satisfy-
ing yβ,γ=yγ,β. We associate it with a linear functional Ly:C[x,x]Cby
f=X
(β,γ)
fβ,γxβxγ7− Ly(f) = X
(β,γ)
fβ,γyβ,γ.
For rN, the r-th order complex moment matrix MC
r(y) is the matrix indexed by
Nn
rsuch that
[MC
r(y)]βγ :=Ly(xβxγ) = yβ,γ,β,γNn
r.
The complex moment matrix MC(y) indexed by Nnis defined similarly. For a self-
conjugate polynomial g=P(β,γ)gβ,γxβxγC[x,x], the r-th order complex
localizing matrix MC
r(gy) associated with gis the matrix indexed by Nn
rsuch that
[MC
r(gy)]βγ :=Ly(gxβxγ) = X
(β,γ)
gβ,γyβ+β,γ+γ,β,γNn
r.
The sequence yis called a complex moment sequence if it can be realized by a
Borel measure µ, i.e., yβ,γ=RKxβxγdµfor any β,γNn, and yis said to be of
finite rank if µis a finitely atomic measure.
4JIE WANG
Lemma 2.2 ([5], Theorem 5.1). If yis a complex moment sequence of finite rank,
then MC(y)0and the rank of yis equal to rank MC(y).
Let d0:= max {|β|,|γ|:fβ,γ= 0},di:= max {|β|,|γ|:gi
β,γ= 0}for i[m],
where f=P(β,γ)fβ,γxβxγ, gi=P(β,γ)gi
β,γxβxγ. Set dmin := max {d0, d1, . . . , dm}.
With rdmin, the complex Lasserre’s hierarchy of moment relaxations for (CPOP)
[5] is given by
(2.6) τr:=
inf
yLy(f)
s.t.MC
r(y)0, y0,0= 1
MC
rdi(giy)0, i [m].
Note that (2.6) is a complex semidefinite program (SDP). To reformulate it as a real
SDP, we refer the reader to [15].
3. Multiplication operators and shift operators. In this section, we estab-
lish an interesting connection between multiplication operators and shift operators.
For pF[x]r(resp. F[x]), we write pfor the coefficient vector of psuch that
p=p[x]r(resp. p=p[x]).
Lemma 3.1 ([8], Lemma 5.2). The kernel I:={pR[x]|MR(y)p=0}of a
moment matrix MR(y)is an ideal in R[x]. Moreover, if MR(y)0, then Iis a real
radical ideal.
Lemma 3.2 ([7]). Let ybe a complex moment sequence of finite rank. The kernel
I:={pC[x]|MC(y)p=0}of the moment matrix MC(y)is a radical ideal in
C[x].
Suppose that yis a (real or complex) moment sequence of rank t. Let I:={p
F[x]|MF(y)p=0}. Then F[x]/I is a linear space over Fof dimension t. The
multiplication operators Mi, i [n] acting on F[x]/I are defined by
Mi:F[x]/I F[x]/I,(3.1)
p7− xip.
Since the moment matrix MF(y) is PSD with rank MF(y) = t, it can be factorized
in the Grammian form such that
(3.2) [MF(y)]βγ =a
βaγ,β,γNn,
where {aα}αNnFt. The shift operators T1, . . . , Tn:FtFtare defined by
(3.3) Ti:X
α
pαaα7− X
α
pαaα+ei,
where {e1,...,en}is the standard vector basis of Nn.
Let us define the following linear map
(3.4) θ:F[x] Ft, p =X
α
pαxα7− X
α
pαaα.
Lemma 3.3. The linear map θinduces an isomorphism: F[x]/I
=Ft.
STRENGTHENING LASSERRE’S HIERARCHY 5
Proof. It is clear that θis surjective. We remain to show that the kernel of θis
I. First, let p=PβpβxβI. It follows from
(3.5) MF(y)p=
X
β
a
αaβpβ
αNn
=0
that a
α(Pβpβaβ) = 0 for all aα. Since {aα}αNnspans Ft, we obtain Pβpβaβ=0.
This proves pker(θ) and hence Iker(θ). Conversely, let pF[x] such that
Pβpβaβ=0. Then we see pI. This proves ker(θ)I.
Theorem 3.4. Let ybe a moment sequence of finite rank. Then the multiplica-
tion operator Miis similar to the shift operator Tifor i[n]. More concretely, we
have Ti=θMiθ1for i[n].
Proof. Let p=PαpαxαF[x]. We have
(3.6) Ti(θ(p)) = Ti X
α
pαaα!=X
α
pαaα+ei.
On the other hand, we have
(3.7) θ(Mi(p)) = θ(xip) = θ X
α
pαxα+ei!=X
α
pαaα+ei.
Thus, Ti(θ(p)) = θ(Mi(p)). It follows Tiθ=θMi. As θis invertible by Lemma
3.3, we obtain Ti=θMiθ1.
Corollary 3.5. Let ybe a moment sequence of finite rank. Then the shift op-
erators T1, . . . , Tnare well-defined.
Proof. We need to show that Ti(Pαpαaα) = 0if Pαpαaα=0. The assumption
Pαpαaα=0implies θ(Pαpαxα) = 0and so PαpαxαI. By Theorem 3.4, we
have
Ti X
α
pαaα!=θMiθ1 X
α
pαaα!
=θMi X
α
pαxα!
=θ X
α
pαxα+ei!
=θ xiX
α
pαxα!=0,
where the last equality follows from the fact that Iis an ideal and so xiPαpαxαI.
For the remainder of the paper, we assume a basis of Rtis given and identify the
shift operators with their representing matrices for convenience.
The real shift operators have the distinguished property of being symmetric.
Lemma 3.6. Let ybe a real moment sequence of finite rank. The shift operators
Ti, i [n]are symmetric.
6JIE WANG
Proof. Suppose that rank MR(y) = tand [MR(y)]βγ =a
βaγfor β,γNn,
where {aα}αNnRt. Let uRtbe arbitrary and we may write
u=X
α
uαaα,
u:= (uα)α.
From
uTiu=X
α,β
uαuβa
α(Tiaβ) = X
α,β
uαuβa
αaβ+ei=
uMR(xiy)
u,
uT
iu=X
α,β
uαuβ(Tiaα)aβ=X
α,β
uαuβa
α+eiaβ=
uMR(xiy)
u,
we obtain u(TiT
i)u= 0. Thus, Ti=T
i.
4. Strengthening Lasserre’s hierarchy. The study of shift operators enables
us to give the following PSD optimality conditions for the pseudo-moment sequence
y.
Theorem 4.1.
(i) Suppose that MR
r(y)0for some rN. Then for any sNwith s < r,
(4.1) MR
s(y)MR
s(xiy)
MR
s(xiy)MR
s(x2
iy)0, i [n].
(ii) Suppose that yis a complex moment sequence admitting a Dirac representing
measure. Then for any sN,
(4.2) MC
s(y)MC
s(xiy)
MC
s(xiy)MC
s(|xi|2y)0, i [n].
Proof. (i). Assume that rank MR
r(y) = tand [MR
r(y)]βγ =a
βaγfor |β|,|γ| r,
where {aα}|α|≤rRt. Let
(4.3) A:={aα}|α|≤s,{aα+ei}|α|≤sRt×2|Nn
s|.
Then one can easily see that
MR
s(y)MR
s(xiy)
MR
s(xiy)MR
s(x2
iy)=AA0,i[n].
(ii). Since yhas a Dirac representing measure, the moment matrix MC
s(y) has rank
one and the shift operators Ti, i [n] are complex numbers. It follows that
(4.4) 1Ti
TiTiTi0, i [n].
Assume that MC(y) = aa, where a= (aα)αNnCNn. For any u= (uα)|α|≤s,v=
(vβ)|β|≤sC|Nn
s|, let u=P|α|≤suαaα, v =P|β|≤svβaβ. We have
uu =X
α,β
uαuβaαaβ=uMC
s(y)u,
uT iv=X
α,β
uαvβTiaαaβ=X
α,β
uαvβaα+eiaβ=uMC
s(xiy)v,
vTiu=X
α,β
vαuβaα(Tiaβ) = X
α,β
vαuβaαaβ+ei=vMC
s(xiy)u,
STRENGTHENING LASSERRE’S HIERARCHY 7
and
vT iTiv=X
α,β
vαvβTiaα(Tiaβ) = X
α,β
vαvβaα+eiaβ+ei=vMC
s(|xi|2y)v,
which gives
(4.5) u vI T i
TiTiTiu
v=uvMC
s(y)MC
s(xiy)
MC
s(xiy)MC
s(|xi|2y)u
v.
From this and (4.4), we obtain (4.2) as desired.
We say that an operator Tis normal if TT=T T . In case that Tis of finite
dimension, it is not hard to see that the normality of Tis equivalent to the PSD
condition TTT T 0, which is further equivalent to
(4.6) I T
T T T0.
Suppose that yis a complex moment sequence such that rank MC(y) = rank MC
s(y).
In a similar manner as the proof of Theorem 4.1 (ii), we can show that the shift
operators Ti, i [n] are normal if and only if the PSD conditions (4.2) hold. It would
be interesting to ask: if yis a complex moment sequence of finite rank, do we have
that the shift operators Ti, i [n] are normal? We will explore this question in the
future work.
Using the PSD optimality conditions in Theorem 4.1, we can strengthen Lasserre’s
hierarchy of moment relaxations. In particular, for real polynomial optimization, we
consider
(4.7) ρ
r:=
inf
yLy(f)
s.t. y0= 1,
MR
rdi(giy)0, i [m],
"MR
r(y)MR
r(xiy)
MR
r(xiy)MR
r(x2
iy)#0, i [n].
Theorem 4.2. It holds ρrρ
rρr+1 fmin for any rdmin.
Proof. Since (4.7) is a strengthening of (2.3), it follows ρrρ
r. The inequality
ρ
rρr+1 follows from the fact that the second PSD constraints of (4.7) are implied
by MR
r+1(y)0 due to Theorem 4.1 (i).
By Theorem 4.2, (4.7) provides an intermediate relaxation between the r-th and
(r+ 1)-th moment relaxations for (RPOP).
For complex polynomial optimization, we consider
(4.8) τ
r,s :=
inf
yLy(f)
s.t.MC
r(y)0, y0,0= 1,
MC
rdi(giy)0, i [m],
"MC
s(y)MC
s(xiy)
MC
s(xiy)MC
s(|xi|2y)#0, i [n].
Here sNis a tunable parameter which we call the normal order.
8JIE WANG
Theorem 4.3. It hold τrτ
r,s τ
r,s+1 fmin and τ
r,s τ
r+1,s for any r
dmin and any sN.
Proof. Since (4.8) is a strengthening of (2.6), it follows τrτ
r. If the infimum of
(CPOP) is attained, let wbe a minimizer of (CPOP) and ybe the moment sequence
of the Dirac measure δw. By Theorem 4.1 (ii), yis a feasible solution of (4.8) and
Ly(f) = fmin. Thus, τ
rfmin. If the infimum of (CPOP) is not attained, let
{w(k)}k1be a minimizing sequence of (CPOP) and y(k)be the moment sequence of
the Dirac measure δw(k), respectively. We have that every y(k)is a feasible solution
of (4.8) and limk→∞ Ly(k)(f) = fmin . Thus, τ
rfmin. The inequalities τ
r,s τ
r,s+1
and τ
r,s τ
r+1,s are easily obtained from the constructions.
By Theorem 4.3, (4.8) is a two-level hierarchy indexed by the relaxation order r
and the normal order s, and hence allows one more level of flexibility by playing with
the two parameters.
5. Integration with sparsity. The strengthening technique discussed in Sec-
tion 4 can be integrated into different sparse versions of Lasserre’s hierarchy to improve
scalability. We refer the reader to [9] for relevant details on different sparse versions
of Lasserre’s hierarchy.
5.1. Correlative sparsity. Consider (RPOP) (resp. (CPOP)). Suppose that
the two index sets [n] and [m] can be decomposed into {I1, . . . , Ip}and {J1, . . . , Jp},
respectively, such that 1) f=f1+· · · +fpwith fkR[xIk] (resp. C[xIk,xIk])
for k[p]; 2) for all k[p] and iJk,giR[xIk] (resp. C[xIk,xIk]), where
R[xIk] (resp. C[xIk,xIk]) denotes the polynomial ring in those variables indexed by
Ik. Let MR
r(y, Ik) (resp. MR
r(gy, Ik)) be the submatrix obtained from MR
r(y) (resp.
MR
r(gy)) by retaining only those rows and columns indexed by βNn
rof MR
r(y)
(resp. MR
r(gy)) with βi= 0 if i /Ik. Then, we can strengthen the correlative
sparse Lasserre’s hierarchy of moment relaxations for real polynomial optimization
by considering
(5.1)
inf
yLy(f)
s.t.MR
r(y, Ik)0, k [p],
MR
rdi(giy, Ik)0, i Jk, k [p],
"MR
1(y)MR
1(xiy)
MR
1(xiy)MR
1(x2
iy)#0, i [n],
y0= 1.
Also, we can strengthen the correlative sparse Lasserre’s hierarchy of moment
relaxations for complex polynomial optimization by considering
(5.2)
inf
yLy(f)
s.t.MC
r(y, Ik)0, k [p],
MC
rdi(giy, Ik)0, i Jk, k [p],
"MC
s(y, Ik)MC
s(xiy, Ik)
MC
s(xiy, Ik)MC
s(|xi|2y, Ik)#0, i Ik, k [p],
y0,0= 1.
5.2. Sign symmetry. For pR[x] and a binary vector s {0,1}n, let [p]s
R[x] be defined by [p]s(x1, . . . , xn):=p((1)s1x1,...,(1)snxn). Then pis said to
STRENGTHENING LASSERRE’S HIERARCHY 9
have the sign symmetry represented by s {0,1}nif [p]s=p. We use S(p) {0,1}n
to denote all sign symmetries of p. Consider (RPOP) and let U:=S(f)Tm
i=1 S(gi).
We define an equivalence relation on [x] by
(5.3) xαxβ US(xα+β).
For each i[m], the equivalence relation gives rise to a partition of [x]rdi:
(5.4) [x]rdi=
pi
G
k=1
[x]rdi,k.
We then build the submatrix MR
rdi,k(giy) of MR
rdi(giy) with respect to the sign
symmetry by retaining only those rows and columns indexed by [x]rdi,k for each
k[pi]. Moreover, for each i[n], the equivalence relation gives rise to a partition
of [x]rxi[x]r: [x]rxi[x]r=Fqi
k=1[x]r,i,k . We build the submatrix NR
r,i,k(y) of
the second PSD matrix in (4.7) by retaining only those rows and columns indexed
by [x]r,i,k for each k[qi]. Then, we can strengthen the sign-symmetry Lasserre’s
hierarchy of moment relaxations for real polynomial optimization by considering
(5.5)
inf
yLy(f)
s.t.MR
rdi,k(giy)0, k [pi],
NR
r,i,k(y)0, k [qi], i [n],
y0= 1.
The complex case proceeds in a similar way, which we omit for conciseness.
6. Numerical experiments. The strengthened real and complex Lasserre’s hi-
erarchies have been implemented in the Julia package TSSOS1. In this section, we
evaluate their performance on diverse polynomial optimization problems using TSSOS
and Mosek 10.0 [1] is employed as an SDP solver with default settings. When pre-
senting the results, ‘LAS’ means the usual Lasserre’s hierarchy and ‘S-LAS’ means
the strengthened Lasserre’s hierarchy; the column labelled by ‘opt’ records optima of
SDPs and the column labelled by ‘time’ records running time in seconds. Moreover,
the symbol ‘-’ means that Mosek runs out of memory. All numerical experiments were
performed on a desktop computer with Intel(R) Core(TM) i9-10900 CPU@2.80GHz
and 64G RAM.
6.1. Minimizing a random real quadratic polynomial with binary vari-
ables. Let us minimize a random real quadratic polynomial with binary variables:
(6.1) (inf
xRn[x]
1Q[x]1
s.t. x2
i= 1, i = 1, . . . , n,
where QR(n+1)×(n+1) is a random symmetric matrix whose entries are selected with
respect to the uniform probability distribution on [0,1]. For each n {10,20,30,40},
we solve three instances using LAS (r= 1,2) and S-LAS (r= 1, s = 1), respectively.
The results are presented in Table 1. For this problem, we empirically observe that
LAS at r= 2 achieves global optimality. It can be seen from the table that the
strengthening technique significantly improves the bound provided by LAS at r= 1
while it is much cheaper than going to LAS at r= 2.
1TSSOS is freely available at https://github.com/wangjie212/TSSOS.
10 JIE WANG
Table 1
Minimizing a random real quadratic polynomial with binary variables.
ntrial LAS (r= 1) LAS (r= 2) S-LAS (r= 1)
opt time opt time opt time
10
1 -6.9868 0.006 -6.6118 0.04 -6.6118 0.03
2 -9.9016 0.006 -9.6732 0.04 -9.6732 0.03
3 -8.6265 0.007 -6.6963 0.04 -6.8216 0.03
20
1 -26.613 0.01 -23.407 5.95 -23.521 0.43
2 -28.474 0.01 -24.330 6.08 -26.575 0.44
3 -30.996 0.01 -27.657 5.61 -27.657 0.47
30
1 -51.429 0.08 -44.597 382 -47.817 6.29
2 -57.277 0.03 -49.871 435 -53.539 5.74
3 -49.950 0.03 -42.548 479 -46.970 5.30
40
1 -79.672 0.09 - - -74.532 43.1
2 -83.814 0.13 - - -81.274 36.9
3 -85.887 0.09 - - -79.748 41.0
6.2. The point cloud registration problem. Given two sets of 3D points
{ai}N
i=1,{bi}N
i=1 with putative correspondences aibi, the point cloud registration
problem in computer vision is to find the best 3D rotation Rand translation tto
align them while explicitly tolerating outliers. It can be formulated as the nonlinear
optimization problem:
(6.2) min
RSO(3),tR3
N
X
i=1
min biRait2
β2
i
,1,
where βi>0 is a given threshold that determines the maximum inlier residual. By
introducing Nbinary variables {θi}N
i=1, (6.2) can be equivalently reformulated as a
polynomial optimization problem:
(6.3) min
RSO(3),tR3,
θi∈{−1,1}
N
X
i=1
1 + θi
2
biRait2
β2
i
+1θi
2.
Note that in (6.3), the rotation matrix Rcan be parametrized by its entries which we
denote by rand the constraint RSO(3) can be expressed by polynomial constraints
in r. Yang and Carlone [20] proposed a customized monomial basis for the dense
Lasserre’s hierarchy for (6.3) which is [1,x,θ,rt,xθ] with x:= [r,t] and θ:=
{θi}N
i=1. Moreover, they also proposed a sparse Lasserre’s hierarchy for (6.3) in which
the variables are decomposed into Ncliques: [x, θi], i [N] and for the i-th clique,
the monomial basis [1,x, θi,rt,xθi] is used. It was empirically shown in [20] that
the dense Lasserre’s hierarchy achieves global optimality at relaxation order r= 2
while the sparse Lasserre’s hierarchy is usually not tight at the same relaxation order.
For each N {10,20,30,40}, we randomly generate three instances of (6.3) with
60% outliers. We solve each instance using the dense LAS (with the above monomial
basis) at r= 2, the sparse LAS (with the above monomial basis) at r= 2, s = 1, and
STRENGTHENING LASSERRE’S HIERARCHY 11
the sparse S-LAS (with the above monomial basis) at r= 2, respectively. The results
are presented in Table 2 from which we can see that the strengthening technique
improves the bound provided by the sparse LAS while it is much cheaper than the
dense LAS.
Table 2
The point cloud registration problem.
Ntrial dense LAS sparse LAS sparse S-LAS
opt time opt time opt time
10
1 6.5437 18.3 6.1294 1.32 6.2392 4.05
2 6.4687 17.8 6.2538 1.33 6.4461 4.56
3 6.3971 21.0 6.1144 1.32 6.2634 4.50
20
1 14.062 424 12.345 2.28 13.007 28.3
2 14.256 350 12.423 2.79 13.053 29.1
3 13.780 321 12.279 2.57 12.851 26.8
30
1 20.870 2461 18.670 3.47 19.696 138
2 20.263 2808 18.522 4.98 19.381 139
3 20.452 2435 18.459 3.64 19.792 136
40
1 - - 24.942 4.84 26.495 662
2 - - 24.783 4.62 26.751 630
3 - - 24.888 4.25 27.295 632
6.3. Minimizing a random complex quadratic polynomial with unit-
norm variables. Let us now minimize a random complex quadratic polynomial with
unit-norm variables:
(6.4) (inf
xCn[x]
1Q[x]1
s.t.|xi|2= 1, i = 1, . . . , n,
where QC(n+1)×(n+1) is a random Hermitian matrix whose entries (both real and
imaginary parts) are selected with respect to the uniform probability distribution on
[0,1]. For each n {10,20,30}, we solve three instances using LAS (r= 1,2) and
S-LAS (r= 1, s = 1), respectively. The results are presented in Table 3. For this
problem, we empirically observe that LAS at r= 2 achieves global optimality. It
is evident from the table that the strengthening technique significantly improves the
bound (indeed, achieving global optimality for n20) provided by LAS at r= 1
while it is much cheaper than going to LAS at r= 2.
6.4. Minimizing a random complex quartic polynomial on a sphere.
Let us minimize a random complex quartic polynomial on a unit sphere:
(6.5) (inf
xCn[x]
2Q[x]2
s.t.|x1|2+· · · +|xn|2= 1,
where QC|[x]2|×|[x]2|(|[x]2|is the cardinality of [x]2) is a random Hermitian matrix
whose entries (both real and imaginary parts) are selected with respect to the uniform
12 JIE WANG
Table 3
Minimizing a random complex quadratic polynomial with unit-norm variables.
ntrial LAS (r= 1) LAS (r= 2) S-LAS (r= 1)
opt time opt time opt time
10
1 -10.830 0.01 -10.474 1.57 -10.474 0.15
2 -14.005 0.01 -13.905 1.76 -13.905 0.15
3 -14.308 0.01 -13.751 1.71 -13.751 0.16
20
1 -39.274 0.03 -38.323 1227 -38.323 6.39
2 -44.009 0.03 -43.911 1076 -43.911 5.51
3 -43.043 0.03 -42.017 1061 -42.017 5.76
30
1 -75.249 0.14 - - -72.948 234
2 -79.995 0.13 - - -79.382 161
3 -74.888 0.12 - - -73.680 148
probability distribution on [0,1]. For each n {5,10,15}, we solve three instances
using LAS (r= 2,3) and S-LAS (r= 2, s = 1), respectively. The results are presented
in Table 4. We can see from the table that the strengthening technique significantly
improves the bound provided by LAS at both r= 2 and r= 3 while it is much cheaper
than going to LAS at r= 3.
Table 4
Minimizing a random complex quartic polynomial on a unit sphere.
ntrial LAS (r= 2) LAS (r= 3) S-LAS (r= 2)
opt time opt time opt time
5
1 -4.4125 0.04 -4.1976 2.09 -4.0517 0.06
2 -2.9632 0.04 -2.5182 1.94 -2.3767 0.05
3 -3.9058 0.04 -3.3651 1.97 -3.1354 0.05
10
1 -5.9950 3.08 - - -4.6231 4.50
2 -5.9757 2.93 - - -4.5794 4.08
3 -5.6221 3.05 - - -4.1087 4.18
15
1 -8.5265 82.3 - - -6.5370 130
2 -8.0241 87.4 - - -6.3118 121
3 -8.0791 85.7 - - -6.1881 123
6.5. Minimizing a random complex quartic polynomial with correlative
sparsity on multi-spheres. Let us minimize a random complex quartic polynomial
with correlative sparsity on multi-spheres:
(6.6) (inf
xCnPl
i=1[xi]
2Qi[xi]2
s.t.xi2= 1, i [l],
where n= 4l+ 2, xi:={x4i3, . . . , x4i+2}, and QiC|[xi]2|×|[xi]2|is a random
Hermitian matrix whose entries (both real and imaginary parts) are selected with
STRENGTHENING LASSERRE’S HIERARCHY 13
respect to the uniform probability distribution on [0,1]. For each l {5,10,50,100},
we solve three instances using the sparse LAS (r= 2,3) and the sparse S-LAS (r=
2, s = 1), respectively. The results are presented in Table 5. Again, we can conclude
from the table that the strengthening technique significantly improves the bound
provided by the sparse LAS at both r= 2 and r= 3 while it is much cheaper than
going to the sparse LAS at r= 3.
Table 5
Minimizing a random complex quartic polynomial on multi-spheres.
ntrial LAS (r= 2) LAS (r= 3) S-LAS (r= 2)
opt time opt time opt time
22
1 -16.561 0.48 -13.190 87.7 -12.911 0.74
2 -17.891 0.47 -14.468 89.5 -13.918 0.72
3 -18.119 0.51 -14.408 90.7 -14.094 0.71
42
1 -34.424 1.28 -27.404 122 -26.607 1.68
2 -35.052 1.30 -28.862 124 -27.896 1.81
3 -34.392 1.24 -27.796 122 -27.071 1.66
202
1 -168.10 5.15 -133.07 645 -132.14 9.81
2 -168.90 5.14 -135.14 597 -133.14 8.34
3 -166.92 4.35 -135.01 612 -132.40 8.89
402
1 -339.50 10.7 - - -268.95 23.5
2 -328.91 12.1 - - -259.32 23.5
3 -333.95 11.0 - - -264.59 21.8
6.6. Smale’s Mean Value conjecture. The following complex polynomial op-
timization problem is borrowed from [17]:
(6.7)
sup
(z,u)Cn+1
|u|
s.t.|H(zi)|≥|u|, i = 1, . . . , n,
z1· · · zn=(1)n
n+1 ,
|z1|2+|z2|2+· · · +|zn|2=n1
n+1 2
n,
where H(y):=1
yRy
0p(z) dzand p(z):= (n+1)(zz1)· · · (zzn) with p(0) = 1. This
problem is used in [17] to verify Smale’s Mean Value conjecture [13, 14] which is open
for n4 since 1981. The optimum of (6.7) is conjectured to be n
n+1 . We refer the
reader to [17] for more details. Here we solve (6.7) with n= 4 using LAS (r= 4,6,8)
and S-LAS (r= 4, s = 1,2,3). The results are presented in Table 6, from which we
see that the strengthening technique enables us to achieve global optimality at lower
relaxation orders so that the computational cost is significantly reduced.
6.7. The Mordell inequality conjecture. Our next example concerns the
Mordell inequality conjecture due to Birch in 1958: if the numbers z1, . . . , znC
satisfies |z1|2+· · · +|zn|2=n, then the maximum of Q1i<jn|zizj|2is nn. This
conjecture was proved for n4 and disproved for n6, and so the only remaining
open case is when n= 5. The reader is referred to [17] for more details. Without
14 JIE WANG
Table 6
The results for (6.7) with n= 4.
LAS
r= 4 r= 6 r= 8
opt time opt time opt time
1.4218 0.16 0.8404 22.8 - -
S-LAS
r= 4, s = 1 r= 4, s = 2 r= 4, s = 3
opt time opt time opt time
1.4218 0.17 1.2727 0.45 0.8000 18.2
loss of generality, we may eliminate one variable and reformulate the conjecture as
the following complex polynomial optimization problem:
(6.8)
sup
zCn1Q1i<jn1|zizj|2Qn1
i=1 |zi+z1+. . . +zn1|2
s.t.|z1|2+· · · +|zn1|2+|z1+. . . +zn1|2=n.
Here we solve (6.8) with n= 3,4 using LAS and S-LAS. The results are presented in
Tables 7 and 8, respectively. From the tables, we see that the strengthening technique
enables us to achieve global optimality at much lower relaxation orders so that the
computational cost is significantly reduced.
Table 7
The results for (6.8) with n= 3.
LAS
r= 4 r= 6 r= 8
opt time opt time opt time
27.347 0.04 27.122 0.19 27.074 0.35
S-LAS
r= 3, s = 0 r= 3, s = 1 r= 3, s = 2
opt time opt time opt time
54.000 0.005 54.000 0.008 27.000 0.01
Table 8
The results for (6.8) with n= 4.
LAS
r= 10 r= 12 r= 14 r= 16 r= 18
opt time opt time opt time opt time opt time
343.66 8.58 326.85 50.1 292.89 212 277.64 790 - -
S-LAS
r= 6, s = 1 r= 6, s = 2 r= 6, s = 3 r= 6, s = 4 r= 6, s = 5
opt time opt time opt time opt time opt time
1638.4 0.13 1337.5 0.20 932.20 0.25 582.86 0.76 256.00 3.10
6.8. The AC-OPF problem. The AC optimal power flow (AC-OPF) is a cen-
tral problem in power systems, which aims to minimize the generation cost of an
alternating current transmission network under the physical constraints. Mathemati-
cally, it can be formulated as the following complex polynomial optimization problem:
STRENGTHENING LASSERRE’S HIERARCHY 15
(6.9)
inf
{Vi}iNPkGc2kSd
ik+Ysh
ik|Vik|2+P(ik,j)EikER
ik
Sikj2
+c1kSd
ik+Ysh
ik|Vik|2+P(ik,j)EikER
ik
Sikj+c0k
s.t. Vref = 0,
Sgl
kSd
ik+Ysh
ik|Vik|2+P(ik,j)EikER
ik
SikjSgu
k,kG,
υl
i |Vi| υu
i,iN,
Sij =Y
ij ibc
ij
2|Vi|2
|Tij |2Y
ij
ViV
j
Tij ,(i, j)E,
Sji =Y
ij ibc
ij
2|Vj|2Y
ij
V
iVj
T
ij ,(i, j)E,
|Sij | su
ij ,(i, j)EER,
θl
ij (ViV
j)θu
ij ,(i, j)E.
For a full description on the AC-OPF problem, the reader may refer to [2] as well as
[16]. For an AC-OPF instance, we can obtain an upper bound (‘ub’) on the optimum
from a local solver. Then the optimality gap between the upper bound and the lower
bound (‘lb’) provided by SDP relaxations is defined by
gap :=ub lb
ub ×100%.
For our purpose, we select instances from the AC-OPF library PGLiB [2] that exhibit
significant optimality gaps. The number appearing in each case name stands for the
number of buses, which is equal to the number of complex variables involved in (6.9).
We solve each instance using the sparse LAS and the sparse S-LAS with minimum
relaxation order [16]. The results are presented in Table 9, from which we see that
the strengthening technique substantially reduce the optimality gap in most cases.
REFERENCES
[1] E. D. Andersen and K. D. Andersen,The Mosek Interior Point Optimizer for Lin-
ear Programming: An Implementation of the Homogeneous Algorithm, in High Perfor-
mance Optimization, vol. 33 of Applied Optimization, Springer US, 2000, pp. 197–232,
https://doi.org/10.1007/978-1-4757-3216-0 8.
[2] S. Babaeinejadsarookolaee, A. Birchfield, R. D. Christie, C. Coffrin, C. De-
Marco, R. Diao, M. Ferris, S. Fliscounakis, S. Greene, R. Huang, et al.,The
power grid library for benchmarking AC optimal power flow algorithms, arXiv preprint
arXiv:1908.02788, (2019).
[3] D. Henrion, M. Korda, and J. B. Lasserre,Moment-sos Hierarchy, The: Lectures In Prob-
ability, Statistics, Computational Geometry, Control And Nonlinear Pdes, vol. 4, World
Scientific, 2020.
[4] D. Henrion and J.-B. Lasserre,Detecting global optimality and extracting solutions in glop-
tipoly, in Positive Polynomials in Control, Springer, 2005, pp. 293–310.
[5] C. Josz and D. K. Molzahn,Lasserre hierarchy for large scale polynomial optimization in
real and complex variables, SIAM Journal on Optimization, 28 (2018), pp. 1017–1048.
[6] J.-B. Lasserre,Global Optimization with Polynomials and the Problem of Moments, SIAM
Journal on Optimization, 11 (2001), pp. 796–817.
[7] J. B. Lasserre, M. Laurent, and P. Rostalski,Semidefinite characterization and computa-
tion of zero-dimensional real radical ideals, Foundations of Computational Mathematics,
8 (2008), pp. 607–647.
[8] M. Laurent,Sums of squares, moment matrices and optimization over polynomials, in Emerg-
ing applications of algebraic geometry, Springer, 2009, pp. 157–270.
[9] V. Magron and J. Wang,Sparse polynomial optimization: theory and practice, World Scien-
tific, 2023.
16 JIE WANG
Table 9
The results for the AC-OPF problem.
case name LAS S-LAS
opt time gap opt time gap
30 as 7.3351e2 0.35 8.66% 7.4426e2 1.13 7.32%
39 epri 1.3749e5 0.27 0.66% 1.3841e5 0.71 0.00%
162 ieee dtc 1.0176e5 13.5 5.84% 1.0647e5 114 1.49%
179 goc 7.5150e5 2.26 0.36% 7.5386e5 7.69 0.05%
30 as sad 8.6569e2 0.33 3.52% 8.7496e2 1.33 2.49%
118 ieee sad 9.6760e4 2.10 7.98% 1.0294e5 5.79 2.10%
162 ieee dtc sad 1.0176e5 10.8 6.36% 1.0738e5 118 1.20%
179 goc sad 7.5279e5 2.43 1.27% 7.5581e5 7.41 0.88%
30 as api 2.6237e3 0.41 47.4% 4.9935e3 1.19 0.05%
39 epri api 2.4511e5 0.23 1.82% 2.4963e5 0.73 0.01%
89 pegase api 1.0139e5 15.2 22.1% 1.0507e5 44.1 19.2%
118 ieee api 1.7571e5 2.01 27.4% 2.2293e5 5.99 7.96%
162 ieee dtc api 1.1526e5 9.81 4.73% 1.1956e5 119 1.17%
179 goc api 1.8603e6 2.74 3.70% 1.9226e6 8.11 0.48%
[10] J. Nie,Tight relaxations for polynomial optimization and lagrange multiplier expressions,
Mathematical Programming, 178 (2019), pp. 1–37.
[11] J. Nie,Moment and Polynomial Optimization, 2023.
[12] C. Riener, T. Theobald, L. J. Andr´
en, and J. B. Lasserre,Exploiting symmetries in sdp-
relaxations for polynomial optimization, Mathematics of Operations Research, 38 (2013),
pp. 122–141.
[13] S. Smale,The fundamental theorem of algebra and complexity theory, Bulletin (New Series)
of the American Mathematical Society, 4 (1981), pp. 1–36.
[14] S. Smale,Mathematical problems for the next century, The mathematical intelligencer, 20
(1998), pp. 7–15.
[15] J. Wang,A more efficient reformulation of complex SDP as real SDP, 2023. Preprint https:
arXiv.org/abs/2307.11599.
[16] J. Wang and V. Magron,Exploiting sparsity in complex polynomial optimization, Journal of
Optimization Theory and Applications, 192 (2022), pp. 335–359.
[17] J. Wang and V. Magron,A real moment-hsos hierarchy for complex polynomial optimization
with real coefficients, arXiv preprint arXiv:2308.14631, (2023).
[18] J. Wang, V. Magron, and J.-B. Lasserre,TSSOS: A moment-SOS hierarchy that exploits
term sparsity, SIAM Journal on Optimization, 31 (2021), pp. 30–58.
[19] J. Wang, V. Magron, J.-B. Lasserre, and N. H. A. Mai,CS-TSSOS: Correlative and term
sparsity for large-scale polynomial optimization, arXiv:2005.02828, (2020).
[20] H. Yang and L. Carlone,One ring to rule them all: Certifiably robust geometric percep-
tion with outliers, in Advances in Neural Information Processing Systems, H. Larochelle,
M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, eds., vol. 33, Curran Associates, Inc.,
2020, pp. 18846–18859.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
In this paper, we study the sparsity-adapted complex moment-Hermitian sum of squares (moment-HSOS) hierarchy for complex polynomial optimization problems, where the sparsity includes correlative sparsity and term sparsity. We compare the strengths of the sparsity-adapted complex moment-HSOS hierarchy with the sparsity-adapted real moment-SOS hierarchy on either randomly generated complex polynomial optimization problems or the AC optimal power flow problem. The results of numerical experiments show that the sparsity-adapted complex moment-HSOS hierarchy provides a trade-off between the computational cost and the quality of obtained bounds for large-scale complex polynomial optimization problems.
Article
Full-text available
The purpose of this work is to present the MOSEK optimizer intended for solution of large-scale sparse linear programs. The optimizer is based on the homogeneous interior-point algorithm which in contrast to the primal-dual algorithm detects a possible primal or dual infeasibility reliably. It employs advanced (parallelized) linear algebra, it handles dense columns in the constraint matrix efficiently, and it has a basis identification procedure. This paper discusses in details the algorithm and linear algebra em­ployed by the MOSEK interior point optimizer. In particular the ho­mogeneous algorithm is emphasized. Furthermore, extensive computa­tional results are reported. These results include comparative results for the XPRESS simplex and the MOSEK interior point optimizer. Fi­nally, computational results are presented to demonstrate the possible speed-up, when using a parallelized version of the MOSEK interior point optimizer on a multiprocessor Silicon Graphics computer.
Article
Full-text available
For an ideal I⊆ℝ[x] given by a set of generators, a new semidefinite characterization of its real radical I(V ℝ(I)) is presented, provided it is zero-dimensional (even if I is not). Moreover, we propose an algorithm using numerical linear algebra and semidefinite optimization techniques, to compute all (finitely many) points of the real variety V ℝ(I) as well as a set of generators of the real radical ideal. The latter is obtained in the form of a border or Gröbner basis. The algorithm is based on moment relaxations and, in contrast to other existing methods, it exploits the real algebraic nature of the problem right from the beginning and avoids the computation of complex components.
Article
Full-text available
In this paper we study various approaches for exploiting symmetries in polynomial optimization problems within the framework of semi definite programming relaxations. Our special focus is on constrained problems especially when the symmetric group is acting on the variables. In particular, we investigate the concept of block decomposition within the framework of constrained polynomial optimization problems, show how the degree principle for the symmetric group can be computationally exploited and also propose some methods to efficiently compute in the geometric quotient.
Article
We propose general notions to deal with large scale polynomial optimization problems and demonstrate their efficiency on a key industrial problem of the twenty first century, namely the optimal power flow problem. These notions enable us to find global minimizers on instances with up to 4,500 variables and 14,500 constraints. First, we generalize the Lasserre hierarchy from real to complex to numbers in order to enhance its tractability when dealing with complex polynomial optimization. Complex numbers are typically used to represent oscillatory phenomena, which are omnipresent in physical systems. Using the notion of hyponormality in operator theory, we provide a finite convergence criterion which generalizes the Curto-Fialkow conditions of the real Lasserre hierarchy. Second, we introduce the multi-ordered Lasserre hierarchy in order to exploit sparsity in polynomial optimization problems (in real or complex variables) while preserving global convergence. It is based on two ideas: 1) to use a different relaxation order for each constraint, and 2) to iteratively seek a closest measure to the truncated moment data until a measure matches the truncated data. Third and last, we exhibit a block diagonal structure of the Lasserre hierarchy in the presence of commonly encountered symmetries.
Article
This paper proposes tight semidefinite relaxations for polynomial optimization. The optimality conditions are investigated. We show that generally Lagrange multipliers can be expressed as polynomial functions in decision variables over the set of critical points. The polynomial expressions can be determined by linear equations. Based on these expressions, new Lasserre type semidefinite relaxations are constructed for solving polynomial optimization. We show that the hierarchy of new relaxations has finite convergence, or equivalently, the new relaxations are tight for a finite relaxation order.
Article
PARTI 1. The main goal of this account is to show that a classical algorithm, Newton's method, with a standard modification, is a tractable method for finding a zero of a complex polynomial. Here, by "tractable" I mean that the cost of finding a zero doesn't grow exponentially with the degree, in a certain statistical sense. This result, our main theorem, gives some theoretical ex­ planation of why certain "fast" methods of equation solving are indeed fast. Also this work has the effect of helping bring the discrete mathematics of complexity theory of computer science closer to classical calculus and geome­
Article
This report is my response.