An elementary proof of the Fritz-John and Karush-Kuhn-Tucker conditions in nonlinear programming.

Full text

Page 1

ERIM REPORT SERIES RESEARCH IN M ANAGEMENT
ERIM Report Series reference num ber
Publication
Num ber of pages
Persistent paper URL
Em ail address corresponding author
Address
ERS-2005-057-LIS
October 2005
7

frenk@few.eur.nl
Erasm us Research Institute of Managem ent (ERIM)
RSM Erasm us University / Erasm us School of Econom ics
Erasm us Universiteit Rotterdam
P.O.Box 1738
3000 DR Rotterdam , The Netherlands
Phone: + 31 10 408 1182
Fax: + 31 10 408 9640
Em ail: info@erim .eur.nl
Internet: www.erim .eur.nl

Bibliographic data and classifications of all the ERIM reports are also available on the ERIM website:
www.erim .eur.nl

An Elementary Proof of the Fritz-John and Karush-Kuhn-
Tucker Conditions in Nonlinear Programming

S.I. Birbil, J. B. G. Frenk and G. J. Still

Page 2

ERASMUS RESEARCH INSTITUTE OF MANAGEMENT

REPORT SERIES
RESEARCH IN MANAGEMENT



ABSTRACT AND KEYWORDS
Abstract
In this note we give an elem entary proof of the Fritz-John and Karush-Kuhn-Tucker conditions
for nonlinear finite dim ensional program m ing problem s with equality and/or inequality
constraints. The proof avoids the im plicit function theorem usually applied when dealing with
equality constraints and uses a generalization of Farkas lem m a and the Bolzano-Weierstrass
property for com pact sets.
Free Keywords
Nonlinear program m ing, Fritz-John conditions, Karush-Kuhn-Tucker conditions
Availability
The ERIM Report Series is distributed through the following platform s:
Academ ic Repository at Erasm us University (DEAR), DEAR ERIM Series Portal
Social Science Research Network (SSRN), SSRN ERIM Series Webpage
Research Papers in Econom ics (REPEC), REPEC ERIM Series Webpage
Classifications
The electronic versions of the papers in the ERIM report Series contain bibliographic m etadata
by the following classification system s:
Library of Congress Classification, (LCC) LCC Webpage
Journal of Econom ic Literature, (JEL), JEL Webpage
ACM Com puting Classification System CCS Webpage
Inspec Classification schem e (ICS), ICS Webpage

Page 3

An Elementary Proof of the Fritz-John and
Karush-Kuhn-Tucker Conditions in Nonlinear Programming
S ¸.˙I. Birbil†, J. B. G. Frenk‡∗and G. J. Still?
†Faculty of Engineering and Natural Sciences, Sabancı University
Orhanli-Tuzla, 34956, Istanbul, Turkey.
sibirbil@sabanciuniv.edu
‡Econometric Institute, Erasmus University Rotterdam
P.O. Box 1738, 3000 DR Rotterdam, The Netherlands.
frenk@few.eur.nl
?Department of Mathematical Sciences, University of Twente
P.O. Box 217, 7500 AE Enschede, The Netherlands.
g.still@math.twente.nl
ABSTRACT.
conditions for nonlinear finite dimensional programming problems with equality and/or inequality con-
straints. The proof avoids the implicit function theorem usually applied when dealing with equality
constraints and uses a generalization of Farkas lemma and the Bolzano-Weierstrass property for com-
pact sets.
In this note we give an elementary proof of the Fritz-John and Karush-Kuhn-Tucker
Keywords. Nonlinear programming. Fritz-John conditions, Karush-Kuhn-Tucker conditions
1 Introduction
LetAbeanm×nmatrixwithrowsa?
0 ≤ i ≤ q some non-affine, continuously differentiable functions. We consider the optimization problem
min{f0(x) : x ∈ FP}, FP:= {x ∈ Rn: a?
and the program including equalities
k, 1 ≤ k ≤ m, b ∈ Rmanm-dimensionalvector, andfi: Rn→ R,
kx ≤ bk,1 ≤ k ≤ m, fi(x) ≤ 0,1 ≤ i ≤ q},
(P)
min{f0(x) : x ∈ FQ}, FQ:= FP∩ {x ∈ Rn: hj(x) = 0,1 ≤ j ≤ r},
where the functions hj: Rn→ R, 1 ≤ j ≤ r, are non-affine and continuously differentiable.
Two basic results covered in every course on nonlinear programming are the Fritz-John (FJ) and
Karush-Kuhn-Tucker (KKT) necessary conditions for the local minimizers of optimization problems (P)
and (Q) [7–9]. Denoting the nonnegative orthant of Rlby Rl
(Q)
+, the FJ necessary conditions for problem
∗Corresponding author.
1
October 10, 2005

Page 4

2. The FJ and KKT conditions for problems (P) and (Q)
(P) are given by the following: If xPis a local minimizer of problem (P), then there exist (see for example
[2, 5]) vectors 0 ?= λ ∈ Rq+1
?q
λifi(xP) = 0, 1 ≤ i ≤ q and νk(a?
For optimization problem (Q) the resulting FJ conditions are as follows: If xQis a local minimizer of
problem (Q), then there exist (see for example [2, 5]) vectors (λ,ν) ∈ Rq+1+m
satisfying
?q
λifi(xQ) = 0, 1 ≤ i ≤ q and νk(a?
If λ0given in conditions (FJP) and (FJQ) can be chosen positive, then the resulting necessary conditions
are called the KKT conditions for problems (P) and (Q), respectively. A sufficient condition for λ0to
be positive is given by a so-called first-order constraint qualification. In the next section we firstly give
an elementary proof of the FJ and KKT conditions for problem (P). Then the same proof is given for
optimization problem (Q) by using a perturbation argument but avoiding the implicit function theorem.
+
and ν ∈ Rm
+satisfying
i=0λi∇fi(xP) +?m
k=1νkak= 0,
kxP− bk) = 0, 1 ≤ k ≤ m.
(FJP)
+
, µ ∈ Rrwith (λ,µ) ?= 0
i=0λi∇fi(xQ) +?r
j=1µj∇hj(xQ) +?m
k=1νkak= 0,
kxQ− bk) = 0, 1 ≤ k ≤ m.
(FJQ)
2 The FJ and KKT conditions for problems (P) and (Q)
For δ > 0 and ¯ x ∈ Rn, let N(¯ x,δ) denote a δ-neighborhood of ¯ x given by
N(¯ x,δ) := {x ∈ Rn: ?x − ¯ x? ≤ δ}.
A vector xPis called a local minimizer of optimization problem (P) (respectively, for optimization prob-
lem (Q) if xP∈ FP(respectively, xP∈ FQ) and there exists some δ > 0 such that f0(xP) ≤ f0(x) for
every x ∈ FP∩ N(xP,δ) (respectively, x ∈ FQ∩ N(xP,δ)).
We introduce the active index sets I(x) := {1 ≤ i ≤ q : fi(x) = 0} and K(x) = {1 ≤ k ≤ m :
a?
kx = bk}, and denote by B(x), the matrix consisting of the corresponding active rows a?
k,k ∈ K(x).
Lemma 2.1 If xPis a local minimizer of problem (P), then max{∇fi(xP)?d : i ∈ I(xP) ∪ {0}} ≥ 0
for every d such that B(xP)d ≤ 0.
Proof. Suppose by contradiction there exists some d0satisfying B(xP)d0≤ 0 and
0 > ∇fi(xP)?d0= limt↓0fi(xP+ td0) − fi(xP)
t
for every i ∈ I(xP) ∪ {0}. By the finiteness of the sets {0,...,q} and {1,...,m} and the continuity of fi
this implies the existence of some t0> 0 satisfying
fi(xP+ td0) < 0,i / ∈ I(xP),fi(xP+ td0) < fi(xP),i ∈ I(xP) ∪ {0},A(xP+ td0) ≤ b
for every 0 < t ≤ t0. Hence the vector xP+ td0belongs to FPand satisfies f0(xP+ td0) < f0(xP)
for every 0 < t ≤ t0. This contradicts that xPis a local minimum.
?
Remark 2.1 If the function f0is pseudo-convex and the functions fi,1 ≤ i ≤ q are strictly pseudo-
convex, then for a feasible xP the reverse implication in Lemma 2.1 also holds and in this result local
2
October 10, 2005

Page 5

2. The FJ and KKT conditions for problems (P) and (Q)
minimizer is replaced by global minimizer. A proof of this will be given at the end of this section. More-
over, if maxBd≤0,?d?=1{∇fi(xP)?d : i ∈ I(xP) ∪ {0}} > 0 and xP feasible, then one can show
that xP is a local minimum of order one ([5]), i.e., there exists some δ > 0 and c > 0 such that
f0(x) − f0(xP) ≥ c?x − xP? for every x ∈ FP∩ N(xP,δ) .
The proof of the FJ conditions for problem (P) will be based on the following generalization of Farkas
lemma ([6]). For completeness, a short proof, using the strong duality result for linear programming, will
be given in the appendix.
Lemma 2.2 Let ∆s⊆ Rs
given vectors, then the following conditions are equivalent:
+be the unit simplex. If B is a p × n matrix and ci∈ Rn,1 ≤ i ≤ s, some
1. For every d ∈ Rnsatisfying Bd ≤ 0 it holds that max1≤i≤sc?
2. There exists some λ ∈ ∆sand µ ∈ Rp
id ≥ 0.
+satisfying?s
i=1λici+ B?µ = 0.
Proof. (FJ conditions for problem (P)) By combining Lemma 2.1 and Lemma 2.2, the FJ conditions
follow.
?
It is well-known that the KKT conditions follow from the FJ conditions under some constraint qualifi-
cation. We say that the Mangasarian-Fromovitz (MF) constraint qualification for problem (P) holds at a
feasible point x if there exists some d0satisfying
B(x)d0≤ 0 and maxi∈I(x){∇fi(x)?d0} < 0
We now show that at a local minimizer xPof problem (P) satisfying the MF constraint qualification, the
KKT conditions must hold.
Proof. (KKT conditions for problem (P)). Assume that λ0= 0 in the FJ conditions. Applying Lemma
2.2 to the FJ conditions with λ0= 0 we obtain that maxi∈I(xP)∇fi(xP)?d ≥ 0 for every B(xP)d ≤ 0.
This contradicts the MF constraint qualification.
?
To prove the FJ and KKT conditions for problem (Q) without using the implicit function theorem
we consider for a local minimizer xQof problem (Q) and δ > 0 appropriately chosen and ? > 0, the
perturbed feasible region
Fδ(?) := FP∩ N(xQ,δ) ∩ {x ∈ Rn: hj(x) ≤ ?,−hj(x) ≤ ?,1 ≤ j ≤ r},
and the associated optimization problem
min{f0(x) + ?x − xQ?2: x ∈ Fδ(?)}.
(Qδ(?))
Since the feasible region is compact a global minimizer xQ(?) exists for problem (Qδ(?)). For these
global minimizers one can show the following result.
Lemma 2.3 For any sequence ?l↓ 0 it follows that liml↑∞xQ(?l) = xQ.
Proof. Let us assume to the contrary that there exists a sequence xQ(?l),l ∈ N which does not converge
to xQ. By ?xQ(?l) − xQ? ≤ δ and the Bolzano-Weierstrass property for compact sets there exists some
subsequence xQ(?l),l ∈ L ⊆ N satisfying
liml↑∞, l∈LxQ(?l) = x ?= xQ.
(2.1)
3
October 10, 2005