Page 1

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 117, No. 3, pp. 607–625, June 2003 ( 2003)

Generalized Invexity and Generalized

Invariant Monotonicity1

X. M. YANG,2X. Q. YANG,3and K. L. TEO4

Communicated by S. Schaible

Abstract.

generalized invariant monotone maps are introduced. Some examples

are given which show that invariant monotonicity and generalized

invariant monotonicity are proper generalizations of monotonicity and

generalized monotonicity. Relationships between generalized invariant

monotonicity and generalized invexity are established. Our results are

generalizations of those presented by Karamardian and Schaible.

In this paper, several kinds of invariant monotone maps and

Key Words.

tone maps, invex functions, generalized invex functions.

Invariant monotone maps, generalized invariant mono-

1. Introduction

Convexity is a common assumption made in mathematical program-

ming. In recent years, there have been increasing attempts to weaken the

convexity condition. Consequently, several classes of (generalized) invex

1This research was partially supported by the National Natural Science Foundation of China,

the key project of the Chinese Ministry of Education (No. 01005), EYTP, and the Research

Committee of Hong Kong Polytechnic University. The authors are thankful to Editor-in-Chief

A. Miele, Associate Editor S. Schaible, an anonymous Associate Editor, and three anonymous

referees for their many valuable comments on an early version of this paper. The authors are

also grateful to Professor B.D. Craven for some discussion on this paper.

2Professor, Department of Mathematics, Chongqing Normal University, Chongqing, China.

Current Address: Department of Applied Mathematics, Hong Kong Polytechnic University,

Hung Hom, Kowloon, Hong Kong, China.

3Associate Professor, Department of Applied Mathematics, Hong Kong Polytechnic Univer-

sity, Hung Hom, Kowloon, Hong Kong, China.

4Professor, Department of Applied Mathematics, Hong Kong Polytechnic University, Hung

Hom, Kowloon, Hong Kong, China.

607

0022-3239?03?0600-0607?0 2003 Plenum Publishing Corporation

Page 2

JOTA: VOL. 117, NO. 3, JUNE 2003

608

functions have been introduced in the literature. More specifically, the con-

cept of invexity was introduced in Ref. 1, where it is shown that the Kuhn-

Tucker conditions are sufficient for (global) optimality of nonlinear pro-

gramming problems under invexity conditions. In Ref. 2, Kaul and Kaur

presented strictly pseudoinvex, pseudoinvex, and quasiinvex functions, and

investigated their applications in nonlinear programming. In Refs. 3–4,

Weir and Mond introduced the concept of preinvex functions and applied

it to the establishment of the sufficient optimality conditions and duality

in (multiobjective) nonlinear programming. In Ref. 5, prepseudoinvex and

prequasiinvex functions were introduced and the relationships between

invexity and generalized convexity were established. In Ref. 6, Mohan and

Neogy showed that, under certain conditions, an invex function is preinvex

and a quasiinvex function is prequasiinvex. More recently, characterizations

and applications of preinvex functions, semistrictly preinvex functions, pre-

quasiinvex functions, and semistrictly prequasiinvex functions were studied

in Refs. 7–9.

A concept closely related to the convexity of a real-valued function is

the monotonicity of a vector-valued function. It is well known that the

convexity of a real-valued function is equivalent to the monotonicity of the

corresponding gradient function. It is worth noting that monotonicity has

played a very important role in the study of the existence and solution

methods of variational inequality problems. An important breakthrough

generalization of this relation was given in Ref. 10 for various pseudo?

quasiconvexities and pseudo?quasimonotonicities. Subsequently, there has

been increasing interest in the study of monotonicity and generalized mono-

tonicity and of their relationships to convexity and generalized convexity;

see Refs. 11–13. On the other hand, some relationships between generalized

invexity and generalized invariant monotonicity were given in Refs. 14–

15 under certain conditions. In particular, the existence of solutions to the

variational-like inequality problem was proven under generalized invariant

monotonicity in Ref. 15.

In this paper, we study generalized invariant monotonicity and its

relationships with generalized invexity. We introduce several types of gen-

eralized invariant monotonicities which are generalizations of the (strict)

monotonicity, (strict) pseudomonotonicity, and quasimonotonicity reported

in Ref. 10. The main purpose of this paper is to establish relations among

generalized invariant monotonicities and generalized invexities. Note that

the conditions assumed in this paper are different from those assumed in

Refs. 14–15. Several examples are given to show that these generalized

invariant monotonicities are proper generalization of the corresponding

generalized monotonicities. Moreover, some examples are also presented to

Page 3

JOTA: VOL. 117, NO. 3, JUNE 2003

609

illustrate the properly inclusive relations among the generalized invariant

monotonicities.

2. Invariant Monotone Maps and Strictly Invariant Monotone Maps

Let Γ be a nonempty subset of ?n, let η be a vector-valued function

from XBX into ?n(X ⊂?n), and let F be a vector-valued function from Γ

into ?n.

Definition 2.1.

to η if there exists an η: ?nB?n→?nsuch that, for any x, y∈Γ and

λ∈[0, 1],

See Refs. 3–4. A set Γ is said to be invex with respect

yCλη(x, y)∈Γ.

Definition 2.2.

pair of points x, y∈Γ,

See Ref. 10. F is said to be monotone on Γ if, for every

(yAx)T(F( y)AF(x))¤0.

Definition 2.3.

invariant monotone on Γ with respect to η if, for every pair of points

x, y∈Γ,

Let Γ be an invex set with respect to η. F is said to be

η(x, y)TF( y)Cη(y, x)TF(x)⁄0.

Remark 2.1.

with η(x, y)GyAx, but the converse is not necessarily true.

Every monotone map is an invariant monotone map

Example 2.1.

Let F and η be maps defined by

F(x)G(1Ccosx1, 1Ccosx2),

xG(x1, x2)∈(0, π?2)B(0, π?2),

η(x, y)G[(sinx1Asiny1)?cosy1, (sinx2Asiny2)?cosy2],

x, y∈(0, π?2)B[0, π?2).

Clearly, F is invariant monotone with respect to η. Let

xG(π?4, π?4),

yG(π?6, π?6).

Page 4

JOTA: VOL. 117, NO. 3, JUNE 2003

610

Then,

(yAx) [F( y)AF(x)]G−(π?6)(13?2A12?2)F0.

Thus, F is not monotone.

Definition 2.4

and let η: XBX→? be a vector-valued function. The function f: Γ→? is

said to be preinvex with respect to η if

f(yCλη(x, y))⁄λf(x)C(1Aλ) f(y),

∀x, y∈Γ and λ∈[0, 1].

The function f is said to be strictly preinvex with respect to η if (1) holds

with strict inequality for any pair of distinct points x and y and for

λ∈(0, 1).

See Refs. 3-4. Let the set Γ be invex with respect to η,

(1)

Assumption A.

f: Γ→?. Then,

f(yCη(x, y))⁄f(x),

Let the set Γ be invex with respect to η, and let

for any x, y∈Γ.

Remark 2.2.

preinvexity with λG1.

Assumption A is just the inequality of the definition of

Assumption C.

and for any λ∈[0, 1],

η(y, yCλη(x, y))G−λη(x, y),

η(x, yCλη(x, y))G(1Aλ)η(x, y).

See Ref.6. Let η: XBX→?n. Then, for any x, y∈?n

Remark 2.3.

We will show that Assumption C holds if

η(x, y)GxAyCo (??xAy??).

In fact, the following two equalities hold:

(i)

η(y, yCλη(x, y))

Gη(y, yCλ(xAyCo (??xAy??))

G−λ(xAyCo (??xAy??))Co (λ(??xAyCo (??xAy??))??)

G−λ[xAyCo (??xAy??)Co (??xAyCo (??xAy??)??)]

G−λ[xAyCo (??xAy??)]

G−λη(x, y);

Page 5

JOTA: VOL. 117, NO. 3, JUNE 2003

611

(ii)

η(x, yCλη(x, y))

Gη(x, yCλ(xAyAo(??xAy??)))

GxAyCλ(xAyCo(??xAy??))Co(??xAyCλ(xAyCo(??xAy??)))

GxAyCλ(xAyCo(??xAy??))Co(??xAy??)

G(1Aλ)(xAyCo(??xAy??))

G(1Aλ)η(x, y).

Thus, Assumption C holds when we take

η(x, y)GxAyCo(??xAy??).

Example 2.2.

Let

fG−?x?,

∀x∈KG[−2, 2],

and let

η(x, y)G?

xAy,

xAy,

−2Ay,

2Ay,

if x¤ 0, y¤0,

if xF 0, yF0,

if xH 0, y⁄0,

if x⁄ 0, yH0.

Then, it is easy to verify that f is invex with respect to η on K and that f

and η satisfy Assumptions A and C. However, f is not convex.

The following theorem shows that the preinvexity of a function is

equivalent to the invariant monotone property of its gradient. This is a

generalization of the convexity of a function and the monotonicity of its

gradient obtained in Ref. 16.

Lemma 2.1.

is invex with respect to η, then ∇f is an invariant monotone with respect to

η.

Let f be differentiable on an open set containing Γ. If f

Proof.

The proof follows readily from the definitions of invexity and

invariant monotonicity.

?

Theorem 2.1.

be differentiable on Γ. Then, f is a preinvex function with respect to η on Γ

if and only if ∇f is invariant monotone with respect to η on Γ and f satisfies

Assumption A.

Let f and η satisfy respectively Assumption C, and let f