Content uploaded by Roberto Andreani
Author content
All content in this area was uploaded by Roberto Andreani
Content may be subject to copyright.
The CPLD condition of Qi and Wei implies the
quasinormality constraint qualification
R. Andreani
∗
J. M. Mart´ınez
†
M. L. Schuverdt
‡
March 1, 2004.
Abstract
The Constant Positive Linear Dependence (CPLD) condition for feasible points
of nonlinear programming problems was introduced by Qi and Wei and used for the
analysis of SQP methods. In the paper where the CPLD was introduced, the authors
conjectured that this condition could be a constraint qualification. This conjecture
is proved in the present paper. Moreover, it will be shown that the CPLD condition
implies the quasinormality constraint qualification, but the reciprocal is not true. Re-
lations with other constraint qualifications will be given.
Key words: Nonlinear Programming, Constraint Qualifications, CPLD condition,
Quasi-normality.
1 Introduction
A constraint qualification is a property of feasible points of nonlinear programming prob-
lems that, when satisfied by a local minimizer, guarantees that the KKT conditions take
place at that point. See, for example, [1]. When a constraint qualification is fulfilled it is
possible to think in terms of Lagrange multipliers and, consequently, efficient algorithms
based on duality ideas can be defined.
From the theoretical point of view it is interesting to find weak constraint qualifications.
On the other hand, it is desirable that the fulfillment of a constraint qualification should
be easy to verify.
∗
Department of Applied Mathematics, IMECC-UNICAMP, University of Campinas, CP 6065, 13081-
970 Campinas SP, Brazil. This author was supported by PRONEX-Optimization 76.79.1008-00, FAPESP
(Grant 01-04597-4) and CNPq. e-mail: andreani@ime.unicamp.br
†
Department of Applied Mathematics, IMECC-UNICAMP, University of Campinas, CP 6065, 13081-
970 Campinas SP, Brazil. This author was supported by PRONEX-Optimization 76.79.1008-00, FAPESP
(Grant 01-04597-4) and CNPq. e-mail: martinez@ime.unicamp.br
‡
Department of Applied Mathematics, IMECC-UNICAMP, University of Campinas, CP 6065, 13081-
970 Campinas SP, Brazil. This author was supported by FAPESP (Grants 01-04597-4 and 02-00832-1).
e-mail: schuverd@ime.unicamp.br
1
The most widely used constraint qualification is the linear independence of the gra-
dients of the active constraints at the given point. However, many weaker constraint
qualification properties exist. For many optimization algorithms convergence theorems
can be proved that say that if a feasible limit point satisfies some constraint qualifica-
tion then the KKT conditions hold at this point. Clearly, the weaker is the constraint
qualification used at such a theorem, the stronger the convergence theorem turns out to
be.
The CPLD condition was introduced by Qi and Wei in [2] and used to analyze SQP
algorithms. Theorem 4.2 of [2] says that if a limit point of a general SQP method satisfies
some conditions that include CPLD, then this point is KKT. However, they did not prove
that the CPLD condition is a constraint qualification. This question was presented as an
open problem in Section 2 of [2]. Observe that, if a solution of the nonlinear programming
problem existed satisfying CPLD (and the other hypotheses of Theorem 4.2 of [2]) but
not satisfying KKT, this solution would be impossible to find by the analyzed algorithm.
Therefore, it is important to prove that CPLD is a constraint qualification, so that the
existence of such minimizers is impossible.
Hestenes ([3], page 296) introduced a very general constraint qualification called quasi-
normality. See, also, ([1], page 337). We will prove that CPLD implies quasinormality and,
thus, CPLD is also a constraint qualification. Moreover, we will show that quasinormality
is strictly weaker than CPLD.
This paper is organized as follows. In Section 2 we state the main definitions. In Sec-
tion 3 we prove that CPLD implies quasinormality. In Section 4 we state the relationships
of CPLD with other constraint qualifications. Conclusions are given in Section 5.
2 Definitions
Let h : IR
n
→ IR
m
and g : IR
n
→ IR
p
be continuosly differentiable. Define the feasible set
X as
X = {x ∈ IR
n
| h(x) = 0, g(x) ≤ 0}.
For all x ∈ X, define the set of indices of the active inequality constraints as
I(x) = {j ∈ {1, . . . , p} | g
j
(x) = 0}
We say that x ∈ X is a MF-nonregular if there exist λ
1
, . . . , λ
m
∈ IR, µ
j
≥ 0 ∀ j ∈ I(x),
P
m
i=1
|λ
i
| +
P
p
j=1
µ
j
6= 0 such that
m
X
i=1
λ
i
∇h
i
(x) +
X
j∈I(x)
µ
j
∇g
j
(x) = 0.
An MF-nonregular point is exactly a point that does not satisfy the Mangasarian-
Fromovitz (MF) constraint qualification. See [4, 5]. Feasible points that are not MF-
nonregular will be called MF-regular.
2
We say that x ∈ X satisfies the CPLD condition (see [2]) if it is MF-regular or, being
MF-nonregular with scalars λ
i
, µ
j
, the set of vectors
{∇h
i
(y) | λ
i
6= 0, i = 1, . . . , m} ∪ {∇g
j
(y) | µ
j
> 0, j ∈ I(x)} (1)
is linearly dependent for all y in some neighborhood (not restricted to feasible points) of x.
In other words, the CPLD condition says that linear dependence of the vectors (1) takes
place whenever the Mangasarian-Fromovitz constraint qualification [4, 5] is not fulfilled
at x. So, the Mangasarian-Fromovitz constraint qualification implies the CPLD condition.
The reciprocal is not true. Consider, for example, the feasible set defined by h
1
(x) = x
1
= 0
and h
2
(x) = x
1
= 0.
We say that x ∈ X with active constraints I(x) satisfies the sequential B-property
related to the scalars
¯
λ
1
, . . . ,
¯
λ
m
, ¯µ
j
, j ∈ I(x) if there exists a sequence of (not necessarily
feasible) points y
k
∈ IR
n
such that lim
k→∞
y
k
= x and, for all i ∈ {1, . . . , m} such that
¯
λ
i
6= 0 and all j ∈ I(x) such that ¯µ
j
6= 0,
¯
λ
i
h
i
(y
k
) > 0 and ¯µ
j
g
j
(y
k
) > 0
for all k ∈ {0, 1, 2, . . .}.
We say that x ∈ X satisfies the quasinormality constraint qualification [1, 3] if it is
MF-regular or, being MF-nonregular with scalars λ
1
, . . . , λ
m
, µ
j
, j ∈ I(x), it does not
satisfy the sequential B-property related to λ
1
, . . . , λ
m
, µ
j
, j ∈ I(x). In other words, the
quasinormality constraint qualification says that the sequential B-property cannot be true
if the Mangasarian-Fromovitz constraint qualification is not fulfilled at x.
Our strategy for proving that CPLD implies quasinormality will consist in showing that
the linear dependence of the vectors (1) implies that the sequential B-property cannot be
true.
3 Main results
The proof that CPLD implies quasinormality needs two preparatory results of multidi-
mensional Calculus. These results are given in Lemmas 1 and 2.
Lemma 1. Let D ⊂ IR
n
be an open set, x ∈ D, F : D → IR
n
, F = (f
1
, . . . , f
n
) ∈ C
1
(D),
F
0
(x) nonsingular. (By the Inverse Function Theorem F is an homeomorphism between
an open neighborhood D
1
of x and F (D
1
) and F
−1
: F (D
1
) → D
1
is well defined.)
Assume that q ∈ {1, . . . , n − 1}, f : D → IR, f ∈ C
1
(D) is such that ∇f(y) is a linear
combination of ∇f
1
(y), . . . , ∇f
q
(y) for all y ∈ D.
Define, for all u ∈ F (D
1
),
ϕ(u) = f[F
−1
(u)]. (2)
Then, for all u ∈ F (D
1
), j = q + 1, . . . , n,
∂ϕ
∂u
j
(u) = 0. (3)
3
Proof. Apply the Chain Rule to (2). 2
Lemma 2. Let f, f
1
, . . . , f
q
: D ⊂ IR
n
→ IR continuously differentiable, x ∈ D, D an
open set.
Assume that ∇f
1
(x), . . . , ∇f
q
(x) are linearly independent and that ∇f(y) is a linear
combination of ∇f
1
(y), . . . , ∇f
q
(y) for all y ∈ D. In particular,
∇f(x) =
q
X
i=1
λ
i
∇f
i
(x). (4)
Then, there exists D
2
⊂ IR
q
, an open neighborhood of (f
1
(x), . . . , f
q
(x)), and a function
ϕ : D
2
→ IR, ϕ ∈ C
1
(D
2
), such that, for all y ∈ D
1
, we have that (f
1
(y), . . . , f
q
(y)) ∈ D
2
and
f(y) = ϕ(f
1
(y), . . . , f
q
(y)).
Moreover, for all i = 1, . . . , q,
λ
i
=
∂ϕ
∂u
i
(f
1
(x), . . . , f
q
(x)). (5)
Proof . Define f
q+1
, . . . , f
n
in such a way that the hypotheses of Lemma 1 hold. Then, ap-
ply (3) and, finally, use the Chain Rule and the linear independence of ∇f
1
(x), . . . , ∇f
q
(x)
to deduce (5). 2
Let us prove now the main result of this paper.
Theorem 1. Assume that X, h, g are as defined in Section 2 and that x ∈ X satisfies the
CPLD condition. Then, x satisfies the quasinormality constraint qualification.
Proof. Assume that x ∈ X satisfies the CPLD condition. If x is MF-regular we are done.
Suppose that x is MF-nonregular. Then, there exist λ
1
, . . . , λ
m
∈ IR, µ
j
≥ 0 ∀j ∈ I(x)
such that
m
X
i=1
|λ
i
| +
X
j∈I(x)
µ
j
> 0 (6)
and
m
X
i=1
λ
i
∇h
i
(x) +
X
j∈I(x)
µ
j
∇g
j
(x) = 0. (7)
Define
I
+
(x) = {i ∈ {1, . . . , m} | λ
i
> 0},
I
−
(x) = {i ∈ {1, . . . , m} | λ
i
< 0},
I
0
(x) = {j ∈ I(x) | µ
j
> 0}.
Then, by (7),
X
i∈I
+
(x)
λ
i
∇h
i
(x) +
X
i∈I
−
(x)
λ
i
∇h
i
(x) +
X
j∈I(x)
µ
j
∇g
j
(x) = 0. (8)
4
By (6), I
+
(x) ∪ I
−
(x) ∪ I
0
(x) 6= ∅.
Assume, first, that I
+
(x) 6= ∅. Let i
1
∈ I
+
(x). Then, by (8),
λ
i
1
∇h
i
1
(x) = −
X
i∈I
+
(x)−{i
1
}
λ
i
∇h
i
(x) −
X
i∈I
−
(x)
λ
i
∇h
i
(x) −
X
j∈I
o
(x)
µ
j
∇g
j
(x). (9)
We consider two possibilities:
(a) ∇h
i
1
(x) = 0;
(b) ∇h
i
1
(x) 6= 0.
If (a) holds, then, by the CPLD condition, ∇h
i
1
(y) = 0 for all y in a neighborhood
of x. So, h
i
1
(y) = 0 for all y in that neighborhood. Therefore, a sequence y
k
→ x such
that h
i
1
(y
k
) > 0 for all k cannot exist. Therefore, the sequential B-property related to
λ
1
, . . . , λ
m
, µ
j
, j ∈ I(x) cannot hold. Thus, quasinormality is fulfilled.
Assume now that (b) holds. Then, by the Caratheodory’s Theorem for Cones (see, for
example, [1], page 689), there exist
I
++
⊂ I
+
(x) − {i
1
}, I
−−
⊂ I
−
(x), I
oo
⊂ I
o
(x)
such that the vectors
{∇h
i
(x)}
i∈I
++
, {∇h
i
(x)}
i∈I
−−
, {∇g
j
(x)}
j∈I
oo
(10)
are linearly independent and
λ
i
1
∇h
i
1
(x) = −
X
i∈I
++
¯
λ
i
∇h
i
(x) −
X
i∈I
−−
¯
λ
i
∇h
i
(x) −
X
j∈I
oo
¯µ
j
∇g
j
(x) (11)
with
¯
λ
i
> 0 ∀ i ∈ I
++
,
¯
λ
i
< 0 ∀ i ∈ I
−−
and
¯µ
j
> 0 ∀ j ∈ I
oo
.
By the linear independence of the vectors (10) and continuity arguments, the vectors
{∇h
i
(y)}
i∈I
++
, {∇h
i
(y)}
i∈I
−−
, {∇g
j
(y)}
j∈I
oo
(12)
are linearly independent for all y in a neighborhood of x. But, by the CPLD condition,
the vectors
λ
i
1
∇h
i
1
(y), {∇h
i
(y)}
i∈I
++
, {∇h
i
(y)}
i∈I
−−
, {∇g
j
(y)}
j∈I
oo
are linearly dependent in a neighborhood of x. Therefore, λ
i
1
∇h
i
1
(y) must be a linear
combination of the vectors (12) for all y in a neighborhood of x.
For simplicity, and without loss of generality, let us write:
I
++
= {1, . . . , m
1
},
5
I
−−
= {m
1
+ 1, . . . , m
2
},
I
oo
= {1, . . . , p
1
},
m
2
+ p
1
= q.
By Lemma 2, there exists a smooth function ϕ defined in a neighborhood of (0, . . . , 0) ∈
IR
q
such that, for all y in a neighborhood of x,
λ
i
1
h
i
1
(y) = ϕ(h
1
(y), . . . , h
m
2
(y), g
1
(y), . . . , g
p
1
(y)). (13)
Now, suppose that {y
k
} is a sequence that converges to x such that
h
i
(y
k
) > 0, i = 1, . . . , m
1
,
h
i
(y
k
) < 0, i = m
1
+ 1, . . . , m
2
,
g
j
(y
k
) > 0, j = 1, . . . , p
1
.
Then, by (5), (11) and (13), for k large enough we must have that λ
i
1
h
i
1
(y
k
) ≤ 0. This
implies that the sequential B-property cannot hold.
The proofs for the cases I
−
(x) 6= ∅ and I
o
(x) 6= ∅ are entirely analogous to this case.
Therefore, CPLD implies quasinormality as we wanted to prove. 2
4 Relations with other constraint qualifications
If a local minimizer of a nonlinear programming problem satisfies CPLD, then, by Theo-
rem 1, it satisfies the quasinormality constraint qualification and, so, it satisfies the KKT
conditions. Therefore, the CPLD condition is a constraint qualification. In this section
we prove some relations of CPLD with other constraint qualifications. The most obvious
question is whether quasinormality implies CPLD. The following counterexample shows
that this is not true.
Counterexample 1. Quasi-normality does not imply CPLD.
Take n = 2, m = 2, p = 0,
h
1
(x
1
, x
2
) = x
2
e
x
1
, h
2
(x
1
, x
2
) = x
2
, x
∗
= (0, 0).
We have that ∇h
1
(x
∗
) = ∇h
2
(x
∗
). So, λ
1
∇h
1
(x
∗
)+λ
2
∇h
2
(x
∗
) = 0 implies that λ
1
= −λ
2
.
The sequential B-property does not hold because the sign of h
1
(x) is the same as the sign
of h
2
(x) for all x. Therefore, x
∗
is quasinormal. However, at every neighborhood of x
∗
there are points where ∇h
1
and ∇h
2
are linearly independent. So, the CPLD condition is
not satisfied at x
∗
.
The Constant Rank constraint qualification (CRCQ) was introduced by Janin [6]. We
say that x ∈ X satisfies the CRCQ if the linear dependence of a subset of gradients of
active (equality or inequality) constraints at x implies that those gradients are linearly
6
dependent for all y on some neighborhood of x. Several well known constraint qualifications
with practical relevance imply CRCQ. For example, if the constraints are defined by affine
functions, the CRCQ is fulfilled. Moreover, if x ∈ X satisfies the CRCQ and each equality
constraint h
i
(x) = 0 is replaced by two inequality constraints (h
i
(x) ≤ 0 and −h
i
(x) ≤ 0)
the CRCQ still holds with the new description of X. Note that the Mangasarian-Fromovitz
constraint qualification does not enjoy this property.
Clearly, CRCQ implies CPLD. Since the Mangasarian-Fromovitz constraint qualifica-
tion also implies CPLD, it is interesting to ask whether the CPLD condition is strictly
weaker than CRCQ and Mangasarian-Fromovitz together. The following counterexample
shows that, in fact, there are exist points that satisfy CPLD but do not satisfy neither
CRCQ nor Mangasarian-Fromovitz.
Counterexample 2. CPLD does not imply CRCQ∨MF.
Take n = 2, m = 0, p = 4, x
∗
= (0, 0),
g
1
(x
1
, x
2
) = x
1
g
2
(x
1
, x
2
) = x
1
+ x
2
2
g
3
(x
1
, x
2
) = x
1
+ x
2
g
4
(x
1
, x
2
) = −x
1
− x
2
.
Since ∇g
3
(x
∗
) + ∇g
4
(x
∗
) = 0, x
∗
does not satisfy the Mangasarian-Fromovitz constraint
qualification.
The gradients ∇g
1
(x
∗
), ∇g
2
(x
∗
) are linearly dependent but in every neighborhood of
x
∗
there are points where ∇g
1
(x), ∇g
2
(x) are linearly independent. Therefore, x
∗
does not
satisfy CRCQ.
Let us show that x
∗
satisfies CPLD. Assume that µ
j
≥ 0, j = 1, . . . , 4,
P
4
j=1
µ
j
> 0,
and
µ
1
∇g
1
(x
∗
) + µ
2
∇g
2
(x
∗
) + µ
3
∇g
3
(x
∗
) + µ
4
∇g
4
(x
∗
) = 0. (14)
If µ
2
= 0, (14) implies that ∇g
1
(x
∗
), ∇g
3
(x
∗
), ∇g
4
(x
∗
) are linearly dependent. So, since
g
1
, g
3
, g
4
are linear, ∇g
1
(x), ∇g
3
(x), ∇g
4
(x) are linearly dependent for all x ∈ IR
2
.
Suppose that µ
2
> 0. It is easy to see that this is impossible unless at least two of
the multipliers µ
1
, µ
3
, µ
4
are positive. But, since three vectors in IR
2
are always linearly
dependent it turns out that the CPLD condition holds.
We say that x ∈ X with active constraints I(x) satisfies the sequential P-property
related to the scalars
¯
λ
1
, . . . ,
¯
λ
m
, ¯µ
j
, j ∈ I(x) if there exists a sequence of (not necessarily
feasible) points y
k
∈ IR
n
such that lim
k→∞
y
k
= x and
m
X
i=1
¯
λ
i
h
i
(y
k
) +
X
j∈I(x)
¯µ
j
g
j
(y
k
) > 0. (15)
We say that x ∈ X satisfies the pseudonormality constraint qualification [7] if it is
MF-regular or, being MF-nonregular with scalars λ
1
, . . . , λ
m
, µ
j
, j ∈ I(x), it does not
satisfy the sequential P-property related to λ
1
, . . . , λ
m
, µ
j
, j ∈ I(x). In other words, the
7
pseudonormality constraint qualification says that the sequential P-property cannot be
true if the Mangasarian-Fromovitz constraint qualification is not fulfilled at x.
Clearly, the sequential B-property implies the sequential P-property. So, pseudonor-
mality implies quasinormality. In the following counterexample we see that CPLD does
not imply pseudonormality.
Counterexample 3. CPLD does not imply pseudonormality.
Take n = 1, m = 0, p = 2, x
∗
= 0,
g
1
(x
1
) = −x
1
g
2
(x
1
) = x
1
+ x
2
1
.
The point x
∗
is not MF-regular because ∇g
1
(x
∗
)+ ∇g
2
(x
∗
) = 0. But g
1
(x
1
)+ g
2
(x
1
) =
x
2
1
> 0 for all x
1
6= x
∗
, therefore the sequential P-property holds. Therefore, x
∗
is not
pseudonormal.
Now, ∇g
1
(x
∗
) 6= 0 6= ∇g
2
(x
∗
) and ∇g
1
(x), ∇g
2
(x) are linearly dependent for all x ∈ IR.
Therefore, x
∗
satisfies the CRCQ. So, x
∗
also satisfies CPLD.
Counterexample 1 shows a situation where quasinormality takes place but CPLD does
not. It is easy to see that, in this example, the point x
∗
is not pseudonormal. In fact, take
λ
1
= 1, λ
2
= −1 and consider the sequence y
k
= (1/k, 1/k), for which it is trivial to see
that (15) is fulfilled. However, the following counterexample shows that pseudonormality
does not imply CPLD.
Counterexample 4. Pseudo-normality does not imply CPLD.
Take n = 2, m = 0, p = 2, x
∗
= (0, 0),
g
1
(x
1
, x
2
) = −x
1
g
2
(x
1
, x
2
) = x
1
− x
2
1
x
2
2
.
Then,
∇g
1
(x
1
, x
2
) =
−1
0
!
and ∇g
2
(x
1
, x
2
) =
1 − 2x
1
x
2
2
−2x
2
1
x
2
!
.
So, for all µ
1
= µ
2
> 0, we have:
µ
1
∇g
1
(0, 0) + µ
2
∇g
1
(0, 0) = µ
1
−1
0
!
+ µ
2
1
0
!
= 0. (16)
However, if µ
1
= µ
2
> 0,
µ
1
g
1
(x
1
, x
2
) + µ
2
g
2
(x
1
, x
2
) = −µ
1
x
2
1
x
2
2
< 0 ∀(x
1
, x
2
) 6= (0, 0).
So, x
∗
is pseudonormal.
On the other hand, the gradients ∇g
1
(x), ∇g
2
(x) are linearly independent if x
1
6= 0 6=
x
2
. This implies that the CPLD condition does not hold.
8
5 Conclusions
The CPLD condition seems to be an useful tool for the analysis of convergence of SQP
methods. Qi and Wei [2] used this concept for studying convergence properties of a
feasible SQP algorithm due to Panier and Tits [8]. Very likely, their methodology can
be applied to other optimization algorithms. This condition is reasonably general, in
the sense that is implied by the classical Mangasarian-Fromovitz constraint qualification,
by the linearity of the constraints, by the constant-rank constraint qualification and by
the linear independence of the gradients of active constraints. These features make it
appropriate to be included as one of the standard constraint qualifications in text books.
Here we proved that the CPLD is, in fact, a constraint qualification, as conjectured in
[2]. However, it is not as general as the quasinormality constraint qualification which, on
the other hand, does not seem to be appropriate for the analysis of SQP-like algorithms.
Convergence results of algorithms associated to weak constraint qualifications are
stronger than convergence results associated to strong constraint qualifications, as linear
independence of gradients. The discovery of the status of different constraint qualifications
opens different theoretical and practical questions related to minimization methods. On
one hand, it is interesting to ask whether the stronger convergence properties are satis-
fied. On the other hand, it is interesting to verify whether the satisfaction of stronger
convergence results has practical consequences.
References
[1] D. P. Bertsekas, Nonlinear Programming, 2nd edition, Athena Scientific, Belmont,
Massachusetts, 1999.
[2] L. Qi and Z. Wei, On the constant positive linear dependence condition and its
application to SQP methods, SIAM Journal on Optimization 10, pp. 963-981 (2000).
[3] M. R. Hestenes, Optimization Theory - The finite-dimensional case, John Wiley and
Sons, New York, 1975.
[4] O. L. Mangasarian and S. Fromovitz. The Fritz-John necessary optimality conditions
in presence of equality and inequality constraints. Journal of Mathematical Analysis
and Applications 17, pp. 37-47 (1967)
[5] R. T. Rockafellar, Lagrange multipliers and optimality. SIAM Review 35, pp. 183-
238 (1993).
[6] R. Janin, Direction derivative of the marginal function in nonlinear programming,
Mathematical Programming Study 21, pp. 127-138 (1984).
[7] D. P. Bertsekas and A. E. Ozdaglar, Pseudonormality and a Lagrange multiplier the-
ory for constrained optimization, Journal of Optimization Theory and Applications
114, pp. 287-343 (2002).
9
[8] E. R. Panier and A. L. Tits, On combining feasibility, descent and superlinear
convergence in inequality constrained optimization, Mathematical Programming 59,
pp. 261-276 (1993).
10
