Nondifferentiable and Two-Level Mathematical Programming
Chapters (15)
This subsection is devoted to introducing notation and definitions that are used throughout the book. Constants, variables and functions are boldfaced to show that they are vectors or vector-valued. Vectors are column oriented unless otherwise specified. For two vectors$$
x = \left( {\begin{array}{*{20}{c}}
{{x_1}} \\
\vdots \\
{{x_n}}
\end{array}} \right),{\text{ }}y = \left( {\begin{array}{*{20}{c}}
{{y_1}} \\
\vdots \\
{{y_n}}
\end{array}} \right)$$
the inner product,\(
\Sigma _{i = 1}^n{x_i}{y_i}
\), of x and y is denoted by xTy. Superscript T stands for the transposition. When x is a row vector, we write the inner product as x • y.
In this chapter we deal with the following basic optimization problem: Among the n-tuples (x
1,…, x
n
) that satisfy in m inequalities g
i
(x
1, …,x
n
) ≦ 0, i = 1,…, m, and ℓ equalities h
i
(x
1,…, x
n
) = 0, i = 1,…, ℓ, find a solution (\(
x_1^*, \ldots ,x_n^*
\)) which minimizes the function f (x
1,…,x
n
).
In this chapter we shall discuss optimality conditions and solutions for nondifferentiable optimization problems — optimization problems with nondifferentiable objective and constraint functions. The Kuhn-Tucker conditions [K6] are well known optimality conditions for nonlinear programming problems consisting of differentiable functions. Here, we shall derive Kuhn-Tucker like conditions for two classes of non-differentiable optimization problems consisting of locally Lipschitz functions and of quasidifferentiable functions.
The simplest type of constrained optimization problem is realized when the functions f(x), g(x), and h(x) in (3.1.1) are all linear in x. The resulting formulation is known as a linear programming (LP) problem and plays a central role in virtually every branch of optimization. Many real situations can be formulated or approximated as LPs, optimal solutions are relatively easy to calculate, and computer codes for solving very large instances consisting of millions of variables and hundreds of thousands of constraints are commercially available. Another attractive feature of linear programs is that various subsidiary questions related, for example, to the sensitivity of the optimal solution to changes in the data and the inclusion of additional variables and constraints can be analyzed with little effort.
In the following nine chapters we study optimization problems whose formulations contain minimization and maximization operations in their description — optimization problems with a two-level structure. In many instances, these problems include optimal-value functions that are not necessarily differentiable and hence difficult to work with. In this chapter we highlight a number of important properties of optimal-value functions that derive from results by Clarke [C9], Gauvin-Dubeau [G4], Fiacco [F3], and Hogan [H12] on differentiable stability analysis for nonlinear programs. These results provide the basis for several computational techniques discussed presently.
In the latter half of this book we are concerned with theories and solution methods for various two-level optimization problems, which are generally called two-level mathematical programs. The purpose of this chapter is to present a unified treatment of various two-level optimization problems in the context of general two-level nonlinear programming. In so doing, we show how several variants fit the general model. As such, one can achieve a unified framework for each individual problem discussed in the following chapters.
The use of primal and dual methods are at the heart of finding solutions to large-scale nonlinear programming problems. Both methods are algorithms of a two-level type where the lower-level decision makers work independently to solve their individual subproblems generated by the decomposition of the original (overall) problem. At the same time, the upper-level decision maker solves his coordinating problem by using the results coming from the lower-level optimizations. These algorithms perform optimization calculations successively by an iterative exchange of information between the two levels.
The min-max problem is a model for decision making under uncertainty. The aim is to minimize the function f (x, y) but the decision maker only has control of the vector x ∈ R
n
. After he selects a value for x, an “opponent” chooses a value for y ∈ R
m
which alternatively can be viewed as a vector of disturbances. When the decision maker is risk averse and has no information about how y will be chosen, it is natural for him to assume the worst. In other words, the second decision maker is completely antagonistic and will try to maximize f (x,y) once x is fixed. The corresponding solution is called the min-max solution and is one of several conservative approaches to decision making under uncertainty. When stochastic information is available for y other approaches might be more appropriate (e.g., see [S4, E3]).
This chapter deals with an optimization problem involving unknown parameters (uncertainty). We consider a decision problem whose objective function is minimized under the condition that a certain performance function should always be less than or equal to a prescribed permissible level (for every value of the unknown parameters). In the case that the set in which the unknown parameters must lie contains an infinite number of elements, we say that the corresponding optimization problem has an infinite number of inequality constraints and call it an infinitely constrained optimization problem.†
This chapter is concerned with a two-level design problem [S23, T1, I4] in which the central system makes a decision on parameter values to be assigned to the subsystems so as to optimize its objective, taking the values of the subsystem performance into account. In the second stage, the subsystems optimize their individual objective functions under the given parameters from the center. Such a problem is mathematically formulated as an optimization problem whose objective and constraint functions include the optimal-value functions obtained from the optimization of the subsystem performance indices.KeywordsDirectional DerivativeConstraint FunctionConstraint QualificationBundle Method
Subsystem PerformanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
In a decentralized system, overall optimization is achieved from a central point of view but each subsystem is invested with considerable power. Such a system is characterized by multiple decision makers in the subsystems, each of whom can exercise strong initiatives at the local level in pursuit of its own goal.
Let f
i
, j = 1, …, N, be the criterion functions that depend on the decision maker’s variable x∈ R
n and opponent’s (or disturbance) variables yi ∈ R
mj, j =1, …, N. In the absence of information about the opponent’s strategies a conservative decision maker would assume that he will experience the worst possible outcome at the hands of his opponents. In such a case, the objective functions to be minimized by the decision maker are defined as follows:
In this chapter, we study a min-max approximation problem with satisfactory conditions as constraints. The underlying formulation contains several functions to be approximated and several error functions to be minimized. Specifically, we are concerned with the problem of realizing (or approximating) the desired characteristics of a device or system being designed. The functions to be approximated measure the desired characteristics, while the error functions are viewed as performance indices and represent the difference between the desired and attainable characteristics of the device or system.
The Stackelberg problem is the most challenging two-level structure that we examine in this book. It has numerous interpretations but originally it was proposed as a model for a leader-follower game in which two players try to minimize their individual objective functions F(x, y) and f (x, y), respectively, subject to a series of interdependent constraints [S28, S27]. Play is defined as sequential and the mood as noncooperative. The decision variables are partitioned between the players in such a way that neither can dominate the other. The leader goes first and through his choice of x ∈ R
n is able to influence but not control the actions of the follower. This is achieved by reducing the set of feasible choices available to the latter. Subsequently, the follower reacts to the leader’s decision by choosing a y ∈ R
m in an effort to minimizes his costs. In so doing, he indirectly affects the leader’s solution space and outcome.
The vast majority of research on two-level programming has centered on the linear Stackelberg game, alternatively known as the linear bilevel programming problem (BLPP). In this chapter we present several of the most successful algorithms that have been developed for this case, and when possible, compare their performance. We begin with some basic notation and a discussion of the theoretical character of the problem.
... Bilevel optimization has found a variety of important applications, including adversarial training [36,37,46], continual learning [32], hyperparameter tuning [3,17], image reconstruction [9], meta-learning [4,23,42], neural architecture search [15,30], reinforcement learning [20,27], and Stackelberg games [48]. More applications about it can be found in [2,8,10,11,12,44] and the references therein. Theoretical properties including optimality conditions of (1) have been extensively studied in the literature (e.g., see [12,13,34,47,50]). ...
... Suppose that Assumptions 3 and 4 hold. Let f * , f ,f , g, D x , D y ,f hi ,f low , f low ,f * ,f * hi , andg hi be defined in (29), (30), (31), (32), (43), (44) and (45), L ∇f1 , L ∇f1 , Lf , L ∇g , Lg and G be given in Assumptions 3 and 4, ε, ρ, µ, x 0 , y 0 and z ǫ be given in Algorithm 6, and ...
... Lemma 6. Suppose that Assumptions 3 and 4 hold. Let D y , Lf , G,f * ,f * hi and B + r be given in (30), (43), (44), (92) and Assumption 4, respectively. Then the following statements hold. ...
In this paper we study a class of unconstrained and constrained bilevel optimization problems in which the lower-level part is a convex optimization problem, while the upper-level part is possibly a nonconvex optimization problem. In particular, we propose penalty methods for solving them, whose subproblems turn out to be a structured minimax problem and are suitably solved by a first-order method developed in this paper. Under some suitable assumptions, an operation complexity of and , measured by their fundamental operations, is established for the proposed penalty methods for finding an -KKT solution of the unconstrained and constrained bilevel optimization problems, respectively. To the best of our knowledge, the methodology and results in this paper are new.
... Bilevel optimization has found a variety of important applications, including adversarial training [36,37,46], continual learning [32], hyperparameter tuning [3,17], image reconstruction [9], meta-learning [4,23,42], neural architecture search [15,30], reinforcement learning [20,27], and Stackelberg games [48]. More applications about it can be found in [2,8,10,11,12,44] and the references therein. Theoretical properties including optimality conditions of (1) have been extensively studied in the literature (e.g., see [12,13,34,47,50]). ...
... Suppose that Assumptions 3 and 4 hold. Let f * , f ,f , g, D x , D y ,f hi ,f low , f low ,f * ,f * hi , andg hi be defined in (29), (30), (31), (32), (43), (44) and (45), L ∇f1 , L ∇f1 , Lf , L ∇g , Lg and G be given in Assumptions 3 and 4, ε, ρ, µ, x 0 , y 0 and z ǫ be given in Algorithm 6, and ...
... Lemma 6. Suppose that Assumptions 3 and 4 hold. Let D y , Lf , G,f * ,f * hi and B + r be given in (30), (43), (44), (92) and Assumption 4, respectively. Then the following statements hold. ...
In this paper we study a class of unconstrained and constrained bilevel optimization problems in which the lower-level part is a convex optimization problem, while the upper-level part is possibly a nonconvex optimization problem. In particular, we propose penalty methods for solving them, whose subproblems turn out to be a structured minimax problem and are suitably solved by a first-order method developed in this paper. Under some suitable assumptions, an \emph{operation complexity} of and , measured by their fundamental operations, is established for the proposed penalty methods for finding an -KKT solution of the unconstrained and constrained bilevel optimization problems, respectively. To the best of our knowledge, the methodology and results in this paper are new.
... Hence, it would be natural to first clarify the expressions of the subdifferentials or first order subdifferentials, to be precise, of this function. These quantities and further properties have been extensively studied in the literature; see, e.g., [4,8,16,17,23,24,35] and references therein. Below, we recall the relevant aspects of these properties while adding some crucial aspects based on the concave-convexity, that we define below. ...
... Proof (i) Equality (3.4) is a well-known result by Danskin [8]. As for (3.5), the maximization case proven in [3] can easily be adapted to our minimization case in (1.2). (ii) As the MFCQ holds at (x, y) withȳ = s(x), and remains persistent in some neighborhood of this point, it is well-known that the formula (3.6) will hold in some neighborhood ofx, given that gph K is compact; see, e.g., [35]. (iii) Start by noting that as in the previous case, the LICQ being persistent near (x, y) for all y ∈ S(x), where it holds, then we have (3.7) from Gauvin and Dubeau [17] given that gph K is compact. ...
... It is important to recall that the compactness assumption on gph K can be relaxed by instead imposing some set-valued-type continuity properties on S (1.3); see, e.g., [4,17,23,24,35]. But for the purpose of simplifying the framework used in this paper, we do not consider such relaxations here. ...
We consider the optimal value function of a parametric optimization problem. A large number of publications have been dedicated to the study of continuity and differentiability properties of the function. However, the differentiability aspect of works in the current literature has mostly been limited to first order analysis, with focus on estimates of its directional derivatives and subdifferentials, given that the function is typically nonsmooth. With the progress made in the last two to three decades in major subfields of optimization such as robust, minmax, semi-infinite and bilevel optimization, and their connection to the optimal value function, there is a need for a second order analysis of the generalized differentiability properties of this function. This could enable the development of robust solution algorithms, such as the Newton method. The main goal of this paper is to provide estimates of the generalized Hessian for the optimal value function. Our results are based on two handy tools from parametric optimization, namely the optimal solution and Lagrange mul-tiplier mappings, for which completely detailed estimates of their generalized derivatives are either well-known or can easily be obtained.
... (ii) Une solution optimale du CBLP est toujours un point extrême de la région admissible S (Shimizu, Ishisuka et Bard (1997)). ...
... (iii) S'il existe un point extrême de la région induite R alors il est un point extrême de la région admissibleS (Shimizu, Ishisuka et Bard (1997)). ...
... Une reformulation équivalente du problème CBLP (voir Shimizu, Ishisuka et Bard (1997)) est basée sur la propriété suivante: si X= R; 1 et Y= R; 2 alors une condition nécessaire pour que la solution (x*, y*) soit une solution optimale du problème CBLP est qu'il existe un vecteur u *tel que (u *,x*, y*) soit une solution optimale du problème suivant: ...
Cette thèse porte sur l’étude de variantes du problème de sac à dos. Nous étudions d’abord des problèmes de sac à dos bi-niveaux pour lesquels nous proposons de nouvelles méthodes hybridant programmation dynamique et méthode de séparations et évaluations. Les expériences numériques montrent que les algorithmes présentés sont robustes et surpassent significativement les algorithmes de la littérature. Des heuristiques itératives hybrides basées sur des relaxations sont également conçues pour le problème du sac à dos multidimensionnel à choix multiples. Ce sont des variantes d’un schéma itératif qui convergent théoriquement vers une solution optimale du problème en résolvant une série de sous-problèmes de petite taille produits en exploitant l’information de relaxations. Nos méthodes sont enrichies par des techniques de fixation et l’ajout de coupes et pseudo-coupes induites par la recherche locale et les relaxations pour réduire l’espace de recherche. Les résultats montrent que ces heuristiques convergent rapidement vers des solutions élites et fournissent 13 nouvelles meilleures valeurs sur un ensemble de 33 instances de la littérature. Enfin, une stratégie d’oscillation combinant la programmation mathématique et des heuristiques constructives et destructives est proposée dans le but de résoudre un problème réel pour la gestion de perturbations dans le domaine aérien. Notre approche a été classée seconde d’un challenge international.
... In the original source (Shimizu et al., 1997) of problem sib 1997 02 (shown in Eq. (4)) as well as in other well-known papers (Bard, 1998;Colson, 2002), it is reported that the optimal solution occurs at (x * , y * ) = (4.0, 4.0) with F * = −12.0 and f * = 4.0. ...
... Therefore, we introduce a variation of problem sib 1997 02, which we name sib 1997 02v. The only difference is in the fourth inner constraint, which is changed from −3x + 2y + 4 ≤ 0 to 3x − 2y − 4 ≤ 0. The feasible region of problem sib 1997 02v coincides with the one shown in Figure 16.1.1 in Shimizu et al. (1997) and the optimal solution of this problem is (x * , y * ) = (4.0, 4.0) with F * = −12.0 and f * = 4.0. ...
... We provide an example of use of the BASBL solver to solve problem sib 1997 01 (originally introduced as Example 15.3.1 in Shimizu et al. (1997)), the formulation of which is presented in Example 4: ...
We describe BASBL, our implementation of the deterministic global optimization algorithm Branch-and-Sandwich for a general class of nonconvex/nonlinear bilevel problems, within the open-source MINOTAUR framework. The solver incorporates the original Branch-and-Sandwich algorithm and modifications proposed in (Paulavičius and Adjiman, J. Glob. Opt., 2019, Submitted). We also introduce BASBLib, an extensive online library of bilevel benchmark problems collected from the literature and designed to enable contributions from the bilevel optimization community. We use the problems in the current release of BASBLib to analyze the performance of BASBL using different algorithmic options and we identify a set of default options that provide good overall performance. Finally, we demonstrate the application of BASBL to a set of flexibility index problems including linear and nonlinear constraints.
... For an introduction to bilevel optimization, we refer the interested reader to the monographs [6][7][8][9]. We emphasize that bilevel optimization problems are, in general, highly irregular and nonconvex optimization problems which are only implicitly given as a closed-form representation of the solution mapping associated with (P(x)) is often not available. ...
... Let k ∈ N be fixed for the moment. Due to [1,Theorem 4.7], y k is an optimal solution of the scalar linear optimization problem min y e Dy Ax k + By ≤ d, Dy ≤ Dy k (9) where e ∈ R q denotes the all-ones vector. The dual of this problem is given by ...
... For a detailed introduction to bilevel optimization, see [3,10,15,18,24,33] and references therein. ...
... Due to the implicit structure of the lower level solution mapping, Dempe et al. [9,19,26,29,33,37] developed the first-order optimality conditions based on the differential properties of the solution mapping under different regularity conditions of the lower level problem. ...
Second-order optimality conditions of the bilevel programming problems are dependent on the second-order directional derivatives of the value functions or the solution mappings of the lower level problems under some regular conditions, which can not be calculated or evaluated. To overcome this difficulty, we propose the notion of the bi-local solution. Under the Jacobian uniqueness conditions for the lower level problem, we prove that the bi-local solution is a local minimizer of some one-level minimization problem. Basing on this property, the first-order necessary optimality conditions and second-order necessary and sufficient optimality conditions for the bi-local optimal solution of a given bilevel program are established. The second-order optimality conditions proposed here only involve second-order derivatives of the defining functions of the bilevel problem. The second-order sufficient optimality conditions are used to derive the Q-linear convergence rate of the classical augmented Lagrangian method.
... This results in an infinite-dimensional bilevel optimal control problem. The concept of bilevel optimization is discussed in [1][2][3][4], while [5][6][7][8] present a comprehensive introduction to optimal control. Bilevel optimal control problems are also studied in [9][10][11][12], for example. ...
... On top of this foundation we introduce our penalty approach (Algorithm 2) in Sect. 4. We show that there exists a choice of the penalty parameter (see Lemma 4.7), for which one can expect to find the solution to the subproblems from Algorithm 1. ...
We present a method to solve a special class of parameter identification problems for an elliptic optimal control problem to global optimality. The bilevel problem is reformulated via the optimal-value function of the lower-level problem. The reformulated problem is nonconvex and standard regularity conditions like Robinson’s CQ are violated. Via a relaxation of the constraints, the problem can be decomposed into a family of convex problems and this is the basis for a solution algorithm. The convergence properties are analyzed. It is shown that a penalty method can be employed to solve this family of problems while maintaining convergence speed. For an example problem, the use of the identity as penalty function allows for the solution by a semismooth Newton method. Numerical results are presented. Difficulties and limitations of our approach to solve a nonconvex problem to global optimality are discussed.
... Quite often, hierarchical decision making appears naturally in real world problems raising in economics, logistics, natural sciences, or engineering. In case where two decision makers are involved, so-called bilevel optimization problems can be used to model and study the underlying applications theoretically and numerically, see Bard, 1998;Dempe, 2002;Dempe, Kalashnikov, Pérez-Valdéz, et al., 2015;Mordukhovich, 2018;Shimizu, Ishizuka, Bard, 1997. For given parameters x ∈ R n 1 , let us consider the parametric optimization problem j(x, y) → min y g(y) ∈ C ...
... On the other hand, bilevel programs are mathematically challenging since they are inherently nonsmooth, nonconvex, and irregular in the sense that a reformulation of the hierarchical model as a single-level program results in surrogate problems which suffer from an inherent lack of smoothness, convexity, and regularity. A detailed introduction to the topic of bilevel programming can be found in the monographs Bard, 1998;Dempe, 2002;Dempe, Kalashnikov, Pérez-Valdéz, et al., 2015;Shimizu, Ishizuka, Bard, 1997. ...
This thesis is concerned with the phenomenon of implicit variables in optimization theory. Roughly speaking, a variable is called implicit whenever it is used to model the feasible set but does not appear in the objective function. At the first glance, such variables seem to be less relevant for the purpose of optimization. First, we provide a theoretical study on optimization problems with implicit variables. Therefore, we rely on a model program which covers several interesting problem classes from optimization theory such as bilevel optimization problems, evaluated multiobjective optimization problems, or optimization problems with cardinality constraints. We start our analysis by clarifying that the interpretation of implicit variables as explicit ones induces additional local minimizers. Afterwards, we study three reasonable stationarity systems of Mordukhovich-stationarity-type for the original problem as well as some comparatively weak associated constraint qualifications. The obtained results are applied to the three example classes mentioned above. Second, we introduce switching- and or-constrained optimization problems. Exploiting the observation that each or-constrained optimization problem can be transferred into a switching-constrained optimization problem with the aid of slack variables, one can interpret or-constrained programs as optimization problems comprising implicit variables. Necessary optimality conditions and constraint qualifications for both problem classes are derived. Furthermore, some approaches for the numerical solution of both problem classes are discussed and results of computational experiments are presented. The shortcomings of implicit variables are highlighted in terms of or-constrained optimization. Third, we study three different scenarios where optimality conditions and constraint qualifications for challenging optimization problems can be constructed while abstaining from the introduction of implicit variables. We start by deriving a generalized version of the linear independence constraint qualification as well as second-order necessary and sufficient optimality conditions for so-called disjunctive optimization problems, which cover several interesting but inherently irregular problem classes like mathematical programs with complementarity, switching, or-, and cardinality constraints. Afterwards, we exploit several different single-level reformulations of standard bilevel optimization problems in order to find first- and second-order sufficient optimality conditions. Finally, we study sequential stationarity and regularity conditions for nonsmooth mathematical problems with generalized equation constraints with the aid of the popular limiting variational analysis. The investigated model problem covers the one we use for the theoretical analysis of implicit variables.
... where the union is taken over the set Λ (x, y) := (λ 1 , · · · , λ m1 ) ∈ R m1 : 0 ∈ ∇ y f (x, y) + i∈I λ i ∇ y g i (x, y) , λ i ≥ 0, λ i g i (x, y) = 0, i ∈ I established by Tanino and Ogawa [12] (see also [11,Theorem 6.6.7] and [3, Theorem 4.2]). As specified in [3], the uniform boundedness requirement on ψ around x is imposed in [11,12] while the proof therein works under the inner semicompactness of the argminimum map. ...
... where the union is taken over the set Λ (x, y) := (λ 1 , · · · , λ m1 ) ∈ R m1 : 0 ∈ ∇ y f (x, y) + i∈I λ i ∇ y g i (x, y) , λ i ≥ 0, λ i g i (x, y) = 0, i ∈ I established by Tanino and Ogawa [12] (see also [11,Theorem 6.6.7] and [3, Theorem 4.2]). As specified in [3], the uniform boundedness requirement on ψ around x is imposed in [11,12] while the proof therein works under the inner semicompactness of the argminimum map. Substituting (12) into (6) , we complete the proof of the theorem. ...
In this work, some reasoning's mistakes in the paper by Kohli (doi:10.3934/jimo.2020114) are highlighted. Furthermore, we correct the flaws, propose a correct formulation of the main result (Theorem 5.1) and give alternative proofs.
... Let ω k n (x) be the globally optimal solution set of problem (3). A trivial but very useful observation is that, for any K n , the globally optimal solution set ψ(x) of original follower's problem (2) can be formulated equivalently as the following set ...
... where ψ 3 (x) is the globally optimal solution set of the following problem min y y 3 1 /3 + y 2 2 + y 2 3 + y 2 4 − x y 1 − 4y 2 − 2y 3 − 2y 4 + 6 s.t. y ∈ [−2, 2] 4 . ...
A new numerical method is presented for bilevel programs with a nonconvex follower’s problem. The basic idea is to piecewise construct convex relaxations of the follower’s problems, replace the relaxed follower’s problems equivalently by their Karush–Kuhn–Tucker conditions and solve the resulting mathematical programs with equilibrium constraints. The convex relaxations and needed parameters are constructed with ideas of the piecewise convexity method of global optimization. Under mild conditions, we show that every accumulation point of the optimal solutions of the sequence approximate problems is an optimal solution of the original problem. The convergence theorems of this method are presented and proved. Numerical experiments show that this method is capable of solving this class of bilevel programs.
... Example Shimizu et al. (1997), see [50], considered the bilevel program (1) with ...
... Example Shimizu et al. (1997), see [50], considered the bilevel program (1) with ...
Covers the main algorithmic approaches to bilevel optimization, including local, global, and heuristic techniques
Discusses established and emerging applications, particularly in data analytics, security, energy, electricity markets, and problems over networks
Includes developments in linear, non-linear, optimistic, pessimistic, and mixed-integer bilevel optimization
... Example Shimizu et al. (1997), see [50], considered the bilevel program (1) with ...
... Example Shimizu et al. (1997), see [50], considered the bilevel program (1) with ...
t This chapter presents the Bilevel Optimization LIBrary of the test problems
(BOLIB–for short), which contains a collection of test problems, with continuous
variables, to help support the development of numerical solvers for bilevel optimization. The library contains 173 examples with 138 nonlinear, 24 linear, and 11
simple bilevel optimization problems. This BOLIB collection is probably the largest
bilevel optimization library of test problems. Moreover, as the library is computationenabled with the MATLAB m-files of all the examples, it provides a uniform basis
for testing and comparing algorithms. The library, together with all the related codes,
is freely available at biopt.github.io/bolib.
... Example Shimizu et al. (1997), see [50], considered the bilevel program (1) with ...
... Example Shimizu et al. (1997), see [50], considered the bilevel program (1) with ...
... Typical applications of bilevel optimization address problems in the context of data science, energy markets, finance, and logistics. For a detailed introduction to the topic, the interested reader is referred to the monographs [8,18,23,58]. ...
Usually, bilevel optimization problems need to be transformed into single-level ones in order to derive optimality conditions and solution algorithms. Among the available approaches, the replacement of the lower-level problem by means of duality relations became popular quite recently. We revisit three realizations of this idea which are based on the lower-level Lagrange, Wolfe, and Mond–Weir dual problem. The resulting single-level surrogate problems are equivalent to the original bilevel optimization problem from the viewpoint of global minimizers under mild assumptions. However, all these reformulations suffer from the appearance of so-called implicit variables, i.e., surrogate variables which do not enter the objective function but appear in the feasible set for modeling purposes. Treating implicit variables as explicit ones has been shown to be problematic when locally optimal solutions, stationary points, and applicable constraint qualifications are compared to the original problem. Indeed, we illustrate that the same difficulties have to be faced when using these duality-based reformulations. Furthermore, we show that the Mangasarian–Fromovitz constraint qualification is likely to be violated at each feasible point of these reformulations, contrasting assertions in some recently published papers.
... Next, we discuss about evolutionary algorithms for bilevel optimization. At this point, we would like to refer the readers to other review papers [50], [58], [166], [91], [151] and books [18], [141], [61], [161], [62] on bilevel optimization. ...
Bilevel optimization is defined as a mathematical program, where an optimization problem contains another optimization problem as a constraint. These problems have received significant attention from the mathematical programming community. Only limited work exists on bilevel problems using evolutionary computation techniques; however, recently there has been an increasing interest due to the proliferation of practical applications and the potential of evolutionary algorithms in tackling these problems. This paper provides a comprehensive review on bilevel optimization from the basic principles to solution strategies; both classical and evolutionary. A number of potential application problems are also discussed. To offer the readers insights on the prominent developments in the field of bilevel optimization, we have performed an automated text-analysis of an extended list of papers published on bilevel optimization to date. This paper should motivate evolutionary computation researchers to pay more attention to this practical yet challenging area.
... For more than 50 years, bilevel optimization is a major field of research in mathematical programming due to numerous underlying applications e.g. in data science, economy, finance, machine learning, or natural sciences, see (Bard 1998;Dempe 2002;Shimizu et al. 1997) for an introduction and Dempe (2020) for a recent survey which presents an overview of contributions in this area. Recently, bilevel optimization turned out to be of particular interest in the context of transportation or energy networks, see e.g. ...
This paper deals with the reconstruction of the desired demand in an optimal control problem, stated over a tree-shaped transportation network which is governed by a linear hyperbolic conservation law. As desired demands typically undergo fluctuations due to seasonality or unexpected events making short-term adjustments necessary, such an approach can exemplary be used for forecasting from past data. We suggest to model this problem as a so-called inverse optimal control problem, i.e., a hierarchical optimization problem whose inner problem is the optimal control problem and whose outer problem is the reconstruction problem. In order to guarantee the existence of solutions in the function space framework, the hyperbolic conservation law is interpreted in weak sense allowing for control functions in Lebesgue spaces. For the computational treatment of the model, we transfer the hierarchical problem into a nonsmooth single-level one by plugging the uniquely determined solution of the inner optimal control problem into the outer reconstruction problem before applying techniques from nonsmooth optimization. Some numerical experiments are presented to visualize various features of the model including different types of noise in the demand and strategies of how to observe the network in order to obtain good reconstructions of the desired demand.
... If the parameters of the objective functions and/or constraints are expressed by fuzzy numbers, then it is a ML MOLP problem with FPs. The analytical solution of this problem belongs to the strong NP hard problems [25]. If we start from the assumption that all DMs are interested in the efficient functioning of the entire business system and that they are prepared to cooperate to achieve a compromise solution when making decisions, then the ML MOLP problem with FPs can be presented as a complex MOLP problem with FPs with several DMs. ...
Making numerous business decisions in complex decentralized organizations can be presented as a ML MOLP problem with FPs expressed as triangular fuzzy numbers. This paper presents a methodology that uses several multi-objective programming methods to solve this problem. The Iskander’s method was used for defuzzification of the objective functions and constraints of the problem and a multi-objective programming method based on the cooperation among decision makers was used to determine the aspired values of the variables controlled by the decision-makers and to obtain the preferred non-dominated solution of the entire problem. The efficiency of the proposed methodology was tested on an example of production planning in a complex decentralized company.
... It has been successfully applied to many application areas, and more recently to data science and machine learning (see e.g., [18,20,30,64]). We refer to the monographs [5,12,15,55], the surveys [11,16] and the references within for more applications and the recent advances in related topics. ...
This paper studies bilevel polynomial optimization in which lower level constraining functions depend linearly on lower level variables. We show that such a bilevel program can be reformulated as a disjunctive program using partial Lagrange multiplier expressions (PLMEs). An advantage of this approach is that branch problems of the disjunctive program are easier to solve. In particular, since the PLME can be easily obtained, these branch problems can be efficiently solved by polynomial optimization techniques. Solving each branch problem either returns infeasibility or gives a candidate local or global optimizer for the original bilevel optimization. We give necessary and sufficient conditions for these candidates to be global optimizers, and sufficient conditions for the local optimality. Numerical experiments are also presented to show the efficiency of the method.
... Bilevel programs capture a wide range of important applications in various fields including Stackelberg games and moral hazard problems in economics ( [29,41]), hyperparameter selection and meta learning in machine learning ( [16, 21-23, 26, 27, 30, 31, 34]). More applications can be found in the monographs [3,12,15,40], the survey on bilevel optimization [11,14] and the references within. ...
In this paper, we present difference of convex algorithms for solving bilevel programs in which the upper level objective functions are difference of convex functions, and the lower level programs are fully convex. This nontrivial class of bilevel programs provides a powerful modelling framework for dealing with applications arising from hyperparameter selection in machine learning. Thanks to the full convexity of the lower level program, the value function of the lower level program turns out to be convex and hence the bilevel program can be reformulated as a difference of convex bilevel program. We propose two algorithms for solving the reformulated difference of convex program and show their convergence to stationary points under very mild assumptions. Finally we conduct numerical experiments to a bilevel model of support vector machine classification.
... Recently, BLPP has been applied to hyperparameter optimization and metalearning in machine learning; see, e.g., [29,18,30,51]. More applications can be found in the monographs [45,4,10,15], the survey on bilevel optimization [17], and the references therein. ...
... This results in an infinite-dimensional bilevel optimal control problem. The concept of bilevel optimization is discussed in [Shimizu, Ishizuka, Bard, 1997;Bard, 1998;Dempe, 2002;Dempe, Kalashnikov, et al., 2015], while [Troutman, 1996;Hinze et al., 2009;Tröltzsch, 2009;Lewis, Vrabie, Syrmos, 2012] present a comprehensive introduction to optimal control. Bilevel optimal control problems are also studied in [Knauer, Büskens, 2010;Fisch et al., 2012;Hatz, 2014;Kalashnikov, Benita, Mehlitz, 2015], for example. ...
We present a method to solve a special class of parameter identification problems for an elliptic optimal control problem to global optimality. The bilevel problem is reformulated via the optimal-value function of the lower-level problem. The reformulated problem is nonconvex and standard regularity conditions like Robinson's CQ are violated. Via a relaxation of the constraints, the problem can be decomposed into a family of convex problems and this is the basis for a solution algorithm. The convergence properties are analyzed. It is shown that a penalty method can be employed to solve this family of problems while maintaining convergence speed. For an example problem, the use of the identity as penalty function allows for the solution by a semismooth Newton method. Numerical results are presented. Difficulties and limitations of our approach to solve a nonconvex problem to global optimality are discussed.
... Consequently, (9) is a semi-infinite optimization problem [32][33][34][35][36] (or more precisely, a standard semi-infinite optimization problem, as opposed to a generalized one). It is well-known that any semi-infinite problem-and, in particular, the monotonic regression problem (9)-can be equivalently rewritten as a bi-level optimization problem [35,37,38], namely, ...
Systematic decision making in engineering requires appropriate models. In this article, we introduce a regression method for enhancing the predictive power of a model by exploiting expert knowledge in the form of shape constraints, or more specifically, monotonicity constraints. Incorporating such information is particularly useful when the available datasets are small or do not cover the entire input space, as is often the case in manufacturing applications. We set up the regression subject to the considered monotonicity constraints as a semi-infinite optimization problem, and propose an adaptive solution algorithm. The method is applicable in multiple dimensions and can be extended to more general shape constraints. It was tested and validated on two real-world manufacturing processes, namely, laser glass bending and press hardening of sheet metal. It was found that the resulting models both complied well with the expert’s monotonicity knowledge and predicted the training data accurately. The suggested approach led to lower root-mean-squared errors than comparative methods from the literature for the sparse datasets considered in this work.
... The issue of cordon pricing scheme is a transportation network optimization problem with user equilibrium constraints [25][26][27][28][29]. Therefore, cordon pricing scheme is a bi-level optimization problem. ...
This paper investigates the issue of spatial equity in the cordon pricing scheme. The spatial inequity is for drivers whose travel destinations are located within the cordon or should travel through it and finally they may face with higher travel time. Thus, to alleviate this inequity, a bi-level multi-objective optimization model is developed. Next, an algorithm is implemented according to the second version of the Strength Pareto Evolutionary Algorithm (SPEA2). Then, the developed model is applied to Sioux Falls network and the results is discussed. The results reveal that it seems reasonable to consider spatial equity as an objective function in cordon pricing. In addition, we can create a sustainable situation for the transportation system by improving spatial inequity with a relatively low reduction in social welfare. Moreover, there are spatial inequity impacts in real networks, which should be considered in the cordon pricing scheme. Furthermore, the developed model can increase the public acceptance of drivers and transportation authorities.
... En kısa yolun uzunluğunu enbüyüklemek, [4]. Ağdan geçen akış miktarını enbüyüklemek [6]. ...
Critical infrastructures are so vital for a country that their destruction or incapacity with war, terrorist attack and natural disasters may have irrecoverable effects on security systems together with other social, economic, and public systems. The protection of these facilities is one of the important problems. This study is one of the facility protection problems to determine which facilities will be protected to ensure the uninterrupted demand of a critical zone.
Different attack types and counter defense types, which were ignored in previous studies, were taken into account in this study. Disrupting the supply-demand balance for critical systems with limited resources is also included in the model. This paper presents two new mathematical models for the defense of critical infrastructure systems. The contribution of our models is considering different types of attacks and defense options and disrupting the supply-demand balance. The first model is RୣIMF and considers different attack and defense types. The second model is SD െ RୣIMF and considers supply-balance disruption. Both models are based on RIMF (r-interdiction median problem with fortification) model in the literature. Due to security reasons, it is not possible to find real data to apply the methodology to a real system in our case, therefore we developed a toy problem to solve the models proposed and discussed the results. RୣIMF model offers solution for problems with unlimited capacity and model SD െ RୣIMF offers a solution to problems with limited capacity.
... The bilevel programming problem has many applications including the principalagent moral hazard problem [28], hyperparameters optimization and meta-learning in machine learning [23,17,25,40]. More applications can be found in [32,3,10,11]. For a comprehensive review, we refer to [13] and the references therein. ...
In this paper, we propose a combined approach with second-order optimality conditions of the lower level problem to study constraint qualifications and optimality conditions for bilevel programming problems. The new method is inspired by the combined approach developed by Ye and Zhu in 2010, where the authors combined the classical first-order and the value function approaches to derive new necessary optimality conditions under weaker conditions. In our approach, we add the second-order optimality condition to the combined program as a new constraint. We show that when all known approaches fail, adding the second-order optimality condition as a constraint makes the corresponding partial calmness condition easier to hold. We also give some discussions on optimality conditions and advantages and disadvantages of the combined approaches with the first-order and the second-order information.
... [29,18,30,51]. More applications can be found in the monographs [45,4,10,15], the survey on bilevel optimization [17] and the references therein. ...
The partial calmness for the bilevel programming problem (BLPP) is an important condition which ensures that a local optimal solution of BLPP is a local optimal solution of a partially penalized problem where the lower level optimality constraint is moved to the objective function and hence a weaker constraint qualification can be applied. In this paper we propose a sufficient condition in the form of a partial error bound condition which guarantees the partial calmness condition. We analyse the partial calmness for the combined program based on the Bouligand (B-) and the Fritz John (FJ) stationary conditions from a generic point of view. Our main result states that the partial error bound condition for the combined programs based on B and FJ conditions are generic for an important setting with applications in economics and hence the partial calmness for the combined program is not a particularly stringent assumption. Moreover we derive optimality conditions for the combined program for the generic case without any extra constraint qualifications and show the exact equivalence between our optimality condition and the one by Jongen and Shikhman given in implicit form. Our arguments are based on Jongen, Jonker and Twilt's generic (five type) classification of the so-called generalized critical points for one-dimensional parametric optimization problems and Jongen and Shikhman's generic local reductions of BLPPs.
... Nash equilibria). For more on this general cases the reader is refereed to [71,126,128]. ...
In this thesis, we propose a new risk management framework for telecommunication networks. This is based on theconcept of Risk Assessment Graphs (RAGs). These graphs contain two types of nodes: access point nodes, or startingpoints for attackers, and asset-vulnerability nodes. The latter have to be secured. An arc in the RAG represents apotential propagation of an attacker from a node to another. A positive weight, representing the propagation difficulty ofan attacker, is associated to each arc. First, we propose a quantitative risk evaluation approach based on the shortestpaths between the access points and the asset-vulnerability nodes. Then, we consider a risk treatment problem, calledProactive Countermeasure Selection Problem (PCSP). Given a propagation difficulty threshold for each pair of accesspoint and asset-vulnerability node, and a set of countermeasures that can be placed on the asset vulnerability nodes, thePCSP consists in selecting the minimum cost subset of countermeasures so that the length of each shortest path froman access point to an asset vulnerability node is greater than or equal to the propagation difficulty threshold.We show that the PCSP is NP-Complete even when the graph is reduced to an arc. Then, we give a formulation of theproblem as a bilevel programming model for which we propose two single-level reformulations: a compact formulationbased on LP-duality, and a path formulation with an exponential number of constraints, obtained by projection. Moreover,we study the path formulation from a polyhedral point of view. We introduce several classes of valid inequalities. Wediscuss when the basic and valid inequalities define facets. We also devise separation routines for these inequalities.Using this, we develop a Branch-and-Cut algorithm for the PCSP along with an extensive computational study. Thenumerical tests show the efficiency of the polyhedral results from an algorithmic point of view.Our framework applies to a wide set of real cases in the telecommunication industry. We illustrate this in several practicaluse cases including Internet of Things (IoT), Software Defined Network (SDN) and Local Area Networks (LANs). We alsoshow the integration of our approach in a web application.
... On the other hand, bilevel programs are mathematically challenging because they are inherently nonsmooth, nonconvex, and irregular in the sense that a reformulation of the hierarchical model as a single-level program results in surrogate problems that have an inherent lack of smoothness, convexity, and regularity. A detailed introduction to the topic of bilevel programming can be found in the monographs (Bard [4], Dempe [13], Dempe et al. [19], Shimizu et al. [49]). ...
This paper is concerned with the derivation of first- and second-order sufficient optimality conditions for optimistic bilevel optimization problems involving smooth functions. First-order sufficient optimality conditions are obtained by estimating the tangent cone to the feasible set of the bilevel program in terms of initial problem data. This is done by exploiting several different reformulations of the hierarchical model as a single-level problem. To obtain second-order sufficient optimality conditions, we exploit the so-called value function reformulation of the bilevel optimization problem, which is then tackled with the aid of second-order directional derivatives. The resulting conditions can be stated in terms of initial problem data in several interesting situations comprising the settings where the lower level is linear or possesses strongly stable solutions.
... The bi-level programming model is introduced in the operations research literature in the early 1970s by Bracken and McGill (1973). Mathematical program with equilibrium constraints (MPEC), or briefly MPEC, is an optimization problem in which the essential constraints are defined by a parametric variational inequality or complementarity system (Dempe, 2002;Lachhwani and Dwivedi, 2018;Shimizu et al., 1997;Vicente and Calamai, 1994). It is an extension of the bilevel programming model (Harker and Pang, 1988;Luo et al., 1996). ...
Emergency response activity relies on transportation networks. Emergency facility location interacts with transportation networks clearly. This review is aimed to provide a combined framework for emergency facility location in transportation networks. The article reveals emergency response activities research clusters, issues, and objectives according to keywords co-occurrence analysis. Four classes of spatial separation models in transportation networks, including distance, routing, accessibility, and travel time are introduced. The stochastic and time-dependent characteristics of travel time are described. Travel time estimation and prediction method, travel time under emergency vehicle preemption, transportation network equilibrium method, and travel time in degradable networks are demonstrated. The emergency facilities location models interact with transportation networks, involving location-routing model, location models embedded with accessibility, location models embedded with travel time, and location models employing mathematical program with equilibrium constraints are reviewed. We then point out the-state-of-art challenges: ilities-oriented, evolution landscape and sequential decision modelling, data-driven optimization approach, and machine learning-based algorithms.
... Bilevel programs capture a wide range of important applications in various fields including Stackelberg games and moral hazard problems in economics ( [29,41]), hyperparameter selection and meta learning in machine learning ( [16,21,22,23,26,27,30,31,34]). More applications can be found in the monographs [3,12,15,40], the survey on bilevel optimization [11,14] and the references within. ...
In this paper, we present difference of convex algorithms for solving bilevel programs in which the upper level objective functions are difference of convex functions, and the lower level programs are fully convex. This nontrivial class of bilevel programs provides a powerful modelling framework for dealing with applications arising from hyperparameter selection in machine learning. Thanks to the full convexity of the lower level program, the value function of the lower level program turns out to be convex and hence the bilevel program can be reformulated as a difference of convex bilevel program. We propose two algorithms for solving the reformulated difference of convex program and show their convergence under very mild assumptions. Finally we conduct numerical experiments to a bilevel model of support vector machine classification.
... Monographs, textbooks and edited volumes on the topic are Bard [131], Dempe [385], Dempe et al. [413], Dempe and Kalashnikov [397], Kalashnikov et al. [710], Migdalas et al. [961], Mesanovic et al. [955], Sakawa [1146], Sakawa and Nishizaki [1151], Shimizu et al. [1200], Stein [1249], Talbi [1270], Xu et al. [1388], Zhang et al. [1458]. Bilevel optimization problems are the topic of a chapter in the monograph [479]. ...
Bilevel optimization problems are hierarchical optimization problems where the feasible region of the so-called upper level problem is restricted by the graph of the solution set mapping of the lower level problem. Aim of this article is to collect a large number of references on this topic, to show the diversity of contributions and to support young colleagues who try to start research in this challenging and interesting field.
... Consequently, (9) is a semi-infinite optimization problem [33][34][35][36][37][38] (or more precisely, a standard semi-infinite optimization problem, as opposed to a generalized one). It is wellknown that the monotonic regression problem (9), just like any other semi-infinite problem, can be equivalently rewritten as a bi-level optimization problem [36,39,40], namely ...
We present a regression method for enhancing the predictive power of a model by exploiting expert knowledge in the form of shape constraints, or more specifically, monotonicity constraints. Incorporating such information is particularly useful when the available data sets are small or do not cover the entire input space, as is often the case in manufacturing applications. We set up the regression subject to the considered monotonicity constraints as a semi-infinite optimization problem, and we propose an adaptive solution algorithm. The method is conceptually simple, applicable in multiple dimensions and can be extended to more general shape constraints. It is tested and validated on two real-world manufacturing processes, namely laser glass bending and press hardening of sheet metal. It is found that the resulting models both comply well with the expert's monotonicity knowledge and predict the training data accurately. The suggested approach leads to lower root-mean-squared errors than comparative methods from the literature for the sparse data sets considered in this work.
... Quite often, hierarchical decision making appears naturally in real world problems raising in economics, logistics, natural sciences, or engineering. In case where two decision makers are involved, so-called bilevel optimization problems can be used to model and study the underlying applications theoretically and numerically, see Bard (1998);Dempe (2002); Dempe et al. (2015); Mordukhovich (2018); Shimizu et al. (1997). For given parameters x ∈ R n 1 , let us consider the parametric optimization problem ...
Implicit variables of a mathematical program are variables which do not need to be optimized but are used to model feasibility conditions. They frequently appear in several different problem classes of optimization theory comprising bilevel programming, evaluated multiobjective optimization, or nonlinear optimization problems with slack variables. In order to deal with implicit variables, they are often interpreted as explicit ones. Here, we first point out that this is a light-headed approach which induces artificial locally optimal solutions. Afterwards, we derive various Mordukhovich-stationarity-type necessary optimality conditions which correspond to treating the implicit variables as explicit ones on the one hand, or using them only implicitly to model the constraints on the other. A detailed comparison of the obtained stationarity conditions as well as the associated underlying constraint qualifications will be provided. Overall, we proceed in a fairly general setting relying on modern tools of variational analysis. Finally, we apply our findings to different well-known problem classes of mathematical optimization in order to visualize the obtained theory.
... Bilevel optimization has broad applications, e.g., the moral hazard model of the principal-agent problem in economics [37], electricity markets and networks [7], facility location and production problem [8], meta learning and hyper-parameter selection in machine learning [18,25,33]. More applications can be found in the monographs [4,13,16,49] and the surveys on bilevel optimization [12,17] and the references therein. ...
This paper studies bilevel polynomial optimization problems. To solve them, we give a method based on polynomial optimization relaxations. Each relaxation is obtained from the Kurash-Kuhn-Tucker (KKT) conditions for the lower level optimization and the exchange technique for semi-infinite programming. For KKT conditions, Lagrange multipliers are represented as polynomial or rational functions. The Moment-SOS relaxations are used to solve the polynomial optimizattion relaxations. Under some general assumptions, we prove the convergence of the algorithm for solving bilevel polynomial optimization problems. Numerical experiments are presented to show the efficiency of the method.
... It is difficult to solve because it is nonconvex and violates the Mangasarian-Fromovitz constraint qualification (MFCQ) at every feasible solution (Scholtes & Stöhr, 1999). The common mathematical techniques for solving the MPEQ include (i) non-smooth penalization method (Li, Yang, Zhu, & Meng, 2012;Scholtes & Stöhr, 1999;Shimizu, Ishizuka, & Bard, 1997;Shimizu & Lu, 1995) which uses the gap function (Hearn, 1982) as a penalty term in the lower level problem and convert to single level optimization problem, (ii) smoothing regularization (Birbil, Fang, & Han, 2004), that uses the smoothing method (C. Chen & Mangasarian, 1995) to approximate the upper level and entropic regularization (Fang & Wu, 1996) to solve complementarity constraints, and (iii) solving the MPEQ as a nonlinear program with direct relaxation of the complementarity constraints (Yin & Lawphongpanich, 2007). ...
One of the major causes of non-recurrent traffic congestion in urban areas is the implementation of transport infrastructure projects on city roads. The seeming ubiquity of work zones in cities causes road user frustration and safety hazards, and public relations problems for the transport agency. For this reason, transport agencies seek strategic ways to not only select urban projects but also schedule them in a manner that minimizes the effort associated with these functions. In other words, they seek to exploit the synergies between the tasks of project selection and project scheduling while duly accommodating the project interdependencies. This study introduces a general framework that simultaneously and optimally selects and schedules urban road projects subject to budgetary constraints over a given planning horizon. The project classes considered in this study are lane addition, new road construction, and road maintenance. Through a mimicry of the classic Stackelberg leader-follower game, this problem is formulated herein as a bi-level program. In the upper level, the leader (transport agency decision-makers) determines an optimal set of projects from a larger pool of candidate projects and decides an optimal schedule for their implementation. In the lower level, the followers (road users) seek to minimize their travel delays based on the two decisions made by the leader in the upper level. The numerical experiments show that if the decision-makers do not consider the peri-implementation capacity reduction, the resulting set of selected projects and their construction schedule can lead to significant travel delay cost for the road users.
... The optimistic problem (P op ) is more tractable as compared to the pessimistic problem; therefore, most of the studies handle the optimistic version of the bilevel optimization problem. For the research books on the optimistic formulation of the bilevel programming problem, one is referred to [5,21]. For more recent results on the topic, the interested reader is referred to the papers [6,17,24] and the references therein. ...
The purpose of this paper is to study the pessimistic version of bilevel programming problems in finite-dimensional spaces. Problems of this type are intrinsically nonsmooth (even for smooth initial data). By using optimal value function, we transform the initial problem into a generalized minimax optimization problem. Using convexificators, first-order necessary optimality conditions are then established. An example that illustrates our findings is also given.
... Results were listed in following table, where we used starting point x 0 = y 0 = 1. 66.95 -12.02 12 0.01 3.76E-10 2.24 1.00 0.00E+0 1.80E-1 3.48 3.52 2 7 65.34 -11.88 15 0.01 2.81E-11 1.66 1.00 2.19E-2 9.25E-2 3.45 3.47 Problem name: ShimizuEtal1997b Source: [46] description: Shimizu et al. 1997 defined one example as follows F(x, y) := 16x 2 + 9y 2Comment: According to[46], x * = 11.25, y * = 5 is the global optimal solution of the problem and x * = 7.2, y * = 12.8 is a local optimal solution. Results were listed in following table, where we used starting point x 0 = y 0 = 1. ...
We consider the standard optimistic bilevel optimization problem, in particular upper-and lower-level constraints can be coupled. By means of the lower-level value function, the problem is transformed into a single-level optimization problem with a penalization of the value function constraint. For treating the latter problem, we develop a framework that does not rely on the direct computation of the lower-level value function or its derivatives. For each penalty parameter, the framework leads to a semismooth system of equations. This allows us to extend the semismooth Newton method to bilevel optimization. Besides global convergence properties of the method, we focus on achieving local superlinear convergence to a solution of the semismooth system. To this end, we formulate an appropriate CD-regularity assumption and derive suffcient conditions so that it is fulfilled. Moreover, we develop conditions to guarantee that a solution of the semismooth system is a local solution of the bilevel optimization problem. Extensive numerical experiments on 124 examples of nonlinear bilevel optimization problems from the literature show that this approach exhibits a remarkable performance, where only a few penalty parameters need to be considered.
... denotes the set of Lagrange multipliers of problem (2.1) [26], which is independent of the optimal solution z of problem (2.1) by linear optimization. ...
A special linear, three-level optimization problem is considered where the reaction of the third-level decision maker influences the objective functions of both decision makers on the first and the second level via its optimal objective function value. For this problem, existence of an optimal solution as well as its computation are investigated.
In this paper, we are concerned with a weak (pessimistic) nonlinear bilevel optimization problem. In a sequential setting, for such a problem, we provide sufficient conditions ensuring the existence of solutions via a regularization and the notion of variational convergence. Unlike the approaches adopted by Aboussoror and Loridan (J Math Anal Appl 254: 348-357, 2001) and Aboussoror (Adv Math Res 1: 83-92, 2002), our approach does not require convexity assumptions and gives an extension from the finite dimensional case to a general topological one. Moreover, it gives an improvement of the result given by Loridan and Morgan (in: Buhler et al. (ed) Operations Research Proceedings of the international Conference on Operations Research 90 in Vienna, Springer Verlag, Berlin 1992).
In treating the various optimization problems described in the previous chapters, we have almost always supposed (with the exception of the characterization of saddle points of the Lagrangian function, in Chap. 8) that the functions involved in the said problems are differentiable or continuously differentiable or twice-continuously differentiable. Starting from the 70s of the last century, the necessity of studying nonsmooth (i.e. nondifferentiable) functions and hence nonsmooth optimization problems, gave rise to a new mathematical theory, called Nonsmooth Analysis (this term was introduced by the Canadian mathematician F. H. Clarke).
A bilevel optimisation problem is an optimisation problem which has a second optimisation problem embedded in its constraints. It aims to model problems and decision processes that are hierarchical, which are problem structures that occur frequently in real-life. Thus, due to the wide range of applications of bilevel problems, there is a strong motivation to solve them. The aim of this thesis is to develop an approach to solving bilevel programs by utilising the less commonly used optimal value reformulation. The work can be split into two main contributions. First, a novel trust-region approach to solving nonlinear bilevel problems is proposed, which solves an exact penalisation of the optimal value reformulation. Second, an application of bilevel programming to the London congestion pricing problem is explored, investigating the application of the proposed trust-region method to solve a bilevel formulation of the road pricing problem. One of the most common approaches to solving a bilevel program is to first transform the problem into a single level program. The most popular way of doing so is by replacing the lower level problem by its Karush-Kuhn Tucker conditions. That being said, the reformulation requires strong assumptions on the bilevel program for it to be equivalent to the original problem. An alternative method to transform the problem into a single level problem is to use the optimal value function of the lower level problem. This problem is known to be fully equivalent. However, due to the difficulties in solving it, approaches in the literature that utilise this reformulation are relatively scarce. Under a regularity condition known as the partial calmness condition, an exact penalty problem can be built from the optimal value reformulation. The first contribution of this thesis is the investigation of solving this exact penalty problem to find local solutions of the associated bilevel problem. A novel trust-region algorithm is proposed to solve it, and convergence analysis is explored. The implementation and performance of the algorithm is investigated via extensive numerical experiments on a large test set of nonlinear bilevel problems. This provides strong numerical results that validate the approach for solving bilevel problems. Based on the results and analysis, the performance and limitations of the algorithms are discussed. The second contribution of this thesis is exploring the application of the aforementioned trust-region method on the bilevel optimisation formulation of the road pricing problem. Road pricing is a method used by governments to alleviate congestion in an overcrowded network. The problem has a hierarchical structure, and therefore naturally forms as a bilevel program. We investigate a case study of the problem to the London congestion zone charge: a fixed cordon road pricing scheme implemented in the center of London. Although successful on initial implementation in 2003, congestion in the city has returned to pre-charge levels. Due to recent advances in technology, the Mayor of London is looking to update the congestion charge to a more sophisticated tolling scheme that can charge for distance, time, emissions or other factors. A formulation of the London congestion problem as a bilevel program is presented, which considers the aims and objectives set out by the current Mayor of London. We then show how the trust-region algorithm can be applied to solve a simplified form of the road pricing model commonly seen in the literature. This is a novel approach to the problem, solving a single level continuous exact penalty problem to find local solutions of the road pricing bilevel model. The performance of the algorithm is tested and verified numerically on three network examples of varying sizes, and the efficiency and robustness of the algorithm is assessed.
In this chapter, we first provide an overview of literature addressing the so-called bilevel optimal control problems which are hierarchical optimization problems with two decision makers where at least one of them has to solve an optimal control problem of ODEs or PDEs. By means of two examples from inverse PDE control, we demonstrate how problem-tailored regularization and relaxation approaches can be used to infer necessary optimality conditions in bilevel optimal control. Finally, we present an algorithm which can be used to solve a class of bilevel optimal control problems to global optimality.KeywordsBilevel optimal controlGlobal optimizationInverse optimal controlOptimality conditionsSolution algorithmMathematics Subject Classification (2020)Primary 49K2049M20; Secondary 90C2690C31
The mathematical modeling of numerous real-world applications results in hierarchical optimization problems with two decision makers where at least one of them has to solve an optimal control problem of ordinary or partial differential equations. Such models are referred to as bilevel optimal control problems. Here, we first review some different features of bilevel optimal control including important applications, existence results, solution approaches, and optimality conditions. Afterwards, we focus on a specific problem class where parameters appearing in the objective functional of an optimal control problem of partial differential equations have to be reconstructed. After verifying the existence of solutions, necessary optimality conditions are derived by exploiting the optimal value function of the underlying parametric optimal control problem in the context of a relaxation approach.
We study optimistic bilevel optimization problems, where we assume the lower-level problem is convex with a nonempty, compact feasible region and satisfies a constraint qualification for all possible upper-level decisions. Replacing the lower-level optimization problem by its first-order conditions results in a mathematical program with equilibrium constraints (MPEC) that needs to be solved. We review the relationship between the MPEC and bilevel optimization problem and then survey the theory, algorithms, and software environments for solving the MPEC formulations.
This chapter presents a self-contained approach of variational analysis and generalized differentiation to deriving necessary optimality conditions in bilevel optimization with Lipschitzian data. We mainly concentrate on optimistic models, although the developed machinery also applies to pessimistic versions. Some open problems are posed and discussed.
This paper is concerned with the derivation of first- and second-order sufficient optimality conditions for optimistic bilevel optimization problems involving smooth functions. First-order sufficient optimality conditions are obtained by estimating the tangent cone to the feasible set of the bilevel program in terms of initial problem data. This is done by exploiting several different reformulations of the hierarchical model as a single-level problem. To obtain second-order sufficient optimality conditions, we exploit the so-called value function reformulation of the bilevel optimization problem, which is then tackled with the aid of second-order directional derivatives. The resulting conditions can be stated in terms of initial problem data in several interesting situations comprising the settings where the lower level is linear or possesses strongly stable solutions.
ResearchGate has not been able to resolve any references for this publication.