Project

Decomposition Methods for Mixed Integer Nonlinear Optimization

Goal: The goals of this project are (i) development of new parallel decomposition methods for deterministic global optimization (ii) development of the new open-source MINLP-solver Decogo and (iii) solving difficult industrial optimization models using the new methods.

Updates
0 new
3
Recommendations
0 new
0
Followers
0 new
33
Reads
6 new
847

Project log

Eligius M. T. Hendrix
added a research item
Energy system optimization models are typically large models which combine sub-models which range from linear to very nonlinear. Column generation (CG) is a classical tool to generate feasible solutions of sub-models, defining columns of global master problems, which are used to steer the search for a global solution. In this paper, we present a new inner approximation method for solving energy system MINLP models. The approach is based on combining CG and the Frank Wolfe algorithm for generating an inner approximation of a convex relaxation and a primal heuristic for computing solution candidates. The features of this approach are: (i) no global branch-and-bound tree is used, (ii) sub-problems can be solved in parallel to generate columns, which do not have to be optimal, nor become available at the same time to synchronize the solution, (iii) an arbitrary solver can be used to solve sub-models, (iv) the approach (and the implementation) is generic and can be used to solve other nonconvex MINLP models. We perform experiments with decentralized energy supply system models with more than 3000 variables. The numerical results show that the new decomposition method is able to compute high-quality solutions and has the potential to outperform state-of-the-art MINLP solvers.
Ivo Nowak
added a research item
In this talk we present Decogo, a generic software framework for solving sparse nonconvex MINLPs, based on decomposition based successive approximation. Similar as Column Generation (CG) algorithms for solving huge crew scheduling problems, Decogo computes a solution candidate of a MINLP by first computing a solution of a convex hull relaxation (CR), which is solved by a novel Frank-Wolfe Column Generation (FW-CG) algorithm. The solution of (CR) is used in a primal heuristic for computing solution candidates of the original problem. We made experiments with energy system planning problems up to 3000 variables. A comparison with the MINLP solver Baron showed that the dual and primal bounds were in most cases better. We experimented also with a block-aggregation approach, which significantly improved the dual bound. Since Decogo is not based on branch-and-bound, it is possible to dynamically refine the optimization model during the solution process, e.g. increasing scenario trees in stochastic programming, adding new options in transportation networks, or refining the discretization of differential equations.
Eligius M. T. Hendrix
added a research item
Inhomogeneous membrane cascade systems have been utilized to purify fructooligosaccharides (FOS). Such a process allows a different setup at every stage of the cascade. Varying the setup at every stage implies an optimization problem related to the selection of the membrane and combinations of operating conditions. This paper solves the optimization problem for an inhomogeneous 3-stage membrane cascade and uses the solution as a design guideline. The optimization problem in the 3-stage membrane cascade design has been formulated as a mixed integer, non-linear programming model and solved using the global optimization solver, BARON. By maximizing the yield repetitively with varying purity requirements, a frontier curve has been constructed. The frontier curve was mapped showing the window of operation. The map guides towards the setup that promotes higher permeation in the feed stage when we switch from high yield to high purity. On the other hand, the setup selection at the bottom stage does not show a clear switch, which indicates that the selection at this stage is less critical.
Eligius M. T. Hendrix
added a research item
Most industrial optimization problems are sparse and can be formulated as block-separable mixed-integer nonlinear programming (MINLP) problems, defined by linking low-dimensional sub-problems by (linear) coupling constraints. This paper investigates the potential of using decomposition and a novel multiobjective-based column and cut generation approach for solving nonconvex block-separable MIN-LPs, based on the so-called resource-constrained reformulation. Based on this approach, two decomposition-based inner-and outer-refinement algorithms are presented and preliminary numerical results with nonconvex MINLP instances are reported.
Eligius M. T. Hendrix
added a research item
Many industrial optimization problems are sparse and can be formulated as block-separable mixed-integer nonlinear programming (MINLP) problems, where low-dimensional sub-problems are linked by a (linear) knapsack-like coupling constraint. This paper investigates exploiting this structure using decomposition and a resource constraint formulation of the problem. The idea is that one outer approximation master problem handles sub-problems that can be solved in parallel. The steps of the algorithm are illustrated with numerical examples which shows that convergence to the optimal solution requires a few steps of solving sub-problems in lower dimension.
Ivo Nowak
added an update
We started a new project:
DADLN "Dynamics and adaptive decomposition of machine learning networks",
(2020-2023, BMBF)
Principal investigator of this project is Ouyang Wu
For this project, we plan a refactory of Decogo for disconnecting the generic solver of Decogo and the Pyomo-based model definition. After the refactory, any model type, including a deep learning machine learning model, can be optimized using Decogo
 
Ivo Nowak
added a research item
In this paper, we present a new multi-tree approach for solving large scale Global Optimization Problems (GOP), called DECOA (Decomposition-based Outer Approximation). DECOA is based on decomposing a GOP into sub-problems, which are coupled by linear constraints. It computes a solution by alternately solving sub- and master-problems using Branch-and-Bound (BB). Since DECOA does not use a single (global) BB-tree, it is called a multi-tree algorithm. After formulating a GOP as a block-separable MINLP, we describe how piecewise linear Outer Approximations (OA) can be computed by reformulating nonconvex functions as a Difference of Convex functions. This is followed by a description of the main- and sub-algorithms of DECOA, including a decomposition-based heuristic for finding solution candidates. Finally, we present preliminary results with MINLPs and conclusions.
Eligius M. T. Hendrix
added a research item
This paper presents a new two-phase method for solving convex mixed-integer nonlinear programming (MINLP) problems, called Decomposition-based Outer Approximation Algorithm (DECOA). In the first phase, a sequence of linear integer relaxed sub-problems (LP phase) is solved in order to rapidly generate a good linear relaxation of the original MINLP problem. In the second phase, the algorithm solves a sequence of mixed integer linear programming sub-problems (MIP phase). In both phases the outer approximation is improved iteratively by adding new supporting hyperplanes by solving many easier sub-problems in parallel. DECOA is implemented as a part of Decogo (Decomposition-based Global Optimizer), a parallel decomposition-based MINLP solver implemented in Python and Pyomo. Preliminary numerical results based on 70 convex MINLP instances up to 2700 variables show that due to the generated cuts in the LP phase, on average only 2–3 MIP problems have to be solved in the MIP phase.
Eligius M. T. Hendrix
added a research item
Most industrial optimization problems are sparse and can be formulated as block-separable mixed-integer nonlinear programming (MINLP) problems, defined by linking low-dimensional sub-problems by (linear) coupling constraints. Decomposition methods solve a block-separable MINLP by alternately solving master problems and sub-problems. In practice, decomposition methods are sometimes the only possibility to compute high-quality solutions of large-scale optimization problems. However, efficient implementations may require expert knowledge and problem-specific features. Recently, there is renewed interest in making these methods accessible to general users by developing generic decomposition frameworks and modelling support. The focus of this chapter is on so-called multi-tree decomposition methods, which iteratively approximate the feasible area without using a single (global) branch-and-bound tree, i.e. branch-and-bound is only used for solving sub-problems. After an introduction, we describe first outer approximation (OA) decomposition methods, including the adaptive, multivariate partitioning (AMP) and the novel decomposition-based outer approximation (DECOA) algorithm . This is followed by a description of multi-tree methods using a reduced master problem for solving large-scale industrial optimization problems. The first method to be described applies parallel column generation (CG) and iterative fixing for solving nonconvex transport optimization problems with several hundred millions of variables and constraints. The second method is based on a novel approach combining CG and compact outer approximation. The last methodology to be discussed is the general Benders decomposition method for globally solving large nonconvex stochastic programs using a reduced mixed-integer programming (MIP) master problem.
Ivo Nowak
added a research item
We present new decomposition-based outer and inner approximazion algorithms for solving block-separable MINLPs
Ivo Nowak
added a research item
Traditional deterministic global optimization methods are often based on a Branch-and-Bound (BB) search tree, which may grow rapidly, preventing the method to find a good solution. Motivated by decomposition-based inner approximation (column generation) methods for solving transport scheduling problems with over 100 million variables, we present two new deterministic decomposition-based successive approximation methods for general modular and/or sparse MINLPs. The first algorithm, called DECOA, is a successive MIP-outer-approximation algorithm based on refining nonconvex polyhedral underestimators of nonlinear functions. The second algorithm, called DIOR, is based on successively improving inner and outer approximations by solving column and cut generation sub-problems using DECOA. Both algorithms are part of Decogo, a new parallel MINLP solver. We describe the basic ideas of both algorithms, and present numerical results with Decogo for instances of the MINLPlib2.
Ivo Nowak
added a research item
Traditional deterministic global optimization methods are often based on a Branch-and-Bound (BB) search tree, which may grow rapidly, preventing the method to find a good solution. Motivated by decomposition-based inner approximation (column generation) methods for solving transport scheduling problems with over 100 million variables, we present two new deterministic decomposition-based successive approximation methods for general modular and/or sparse MINLPs. The first algorithm, called DECOA, is a successive MIP-outer-approximation algorithm based on refining nonconvex polyhedral underestimators of nonlinear functions. The second algorithm, called DIOR, is based on successively improving inner and outer approximations by solving column and cut generation sub-problems using DECOA. Both algorithms are part of Decogo, a new parallel MINLP solver. We describe the basic ideas of both algorithms, and present numerical results with Decogo for instances of the MINLPlib2.
Eligius M. T. Hendrix
added a research item
Los algoritmos de diseño de mezclas tie-nen como objetivo determinar las mezclas de materias primas que se ajustan a las restricciones de diseño impuestas para el producto en cuestión. Estas restricciones pueden ser lineales y/o cuadráticas. Las mezclas deben ser óptimas, tanto el coste como el número de materias primas empleado tiene que ser mínimo. Los fabricantes elaboran una serie de productos a partir de un conjunto dado de materias primas. La escasez en la disponibilidad de materias primas introduce restricciones de disponibilidad que alteran la solución Pareto´ optima. Los autores han desarrollado algoritmos de Ramificación y Acotación para resolver problemas de mezcla en donde la complejidad computacional se incrementa con la dimensión del producto. Debido a esta complejidad, se abordará el problema de mezcla para la obtención de sólo dos productos. El diseño de mezclas para dos productos es más difícil que para un único producto porque además de que el diseño de cada producto está sometido a unas restricciones, el frente de Pareto así como la disponibilidad de materias primas pasa a ser común a ambos productos. Se debe realizar una combinación final entre todas las soluciones del primer y el segundo producto para eliminar las combinaciones de mezclas que no satisfacen los criterios impuestos. El conjun-to resultante puede ser usado como dato de entrada del mismo algoritmo cuando se requieran resultados más precisos. El coste computacional de la fase de combinación dependerá del número de elementos del conjunto final de cada producto. Aquí, estudiaremos el coste computacional de las diferentes fases del algoritmo de mezcla para dos pro-ductos y proporcionaremos versiones hebradas para las fases más costosas. Los experimentos se han realizado en una máquina de ocho núcleos con memoria compartida, usando un problema de tamaño medio para evitar largos tiempos de ejecución. Los experimentos muestran que la computación paralela es una herramienta necesaria para hacer una búsqueda ex-haustiva en problemas de grandes dimensiones y de más de un producto.
Ivo Nowak
added a research item
Traditional deterministic global optimization methods are often based on a Branch-and-Bound (BB) search tree, which may grow rapidly, preventing the method to find a good solution. Motivated by decomposition-based inner approximation (column generation) methods for solving transport scheduling problems with over 100 million variables, we present a new deterministic decomposition-based successive approximation method for general modular and/or sparse MINLPs. The new method, called Decomposition-based Inner- and Outer-Refinement, is based on a block-separable reformulation of the model into sub-models. It generates inner- and outer-approximations using column generation, which are successively refined by solving many easier MINLP and MIP subproblems in parallel (using BB), instead of searching over one (global) BB search tree. We present preliminary numerical results with Decogo (Decomposition-based Global Optimizer), a new parallel decomposition MINLP solver implemented in Python and Pyomo.
Ivo Nowak
added a research item
The Topology Optimization (TO) problem is a basic engineering problem of distributing a limited amount of material in a design space. The finite element formulatuation of TO is a large scale nonconvex Mixed Integer Quadratically Constrained Quadratic Program (MIQQP) with millions of variables. Traditional methods for MIQQP are branch-and-bound methods, and cannot solve realistic TO problems. Commercial TO solvers can only compute local solutions, which depend strongly on the starting point. In this talk we present a new decomposition-based global optimization method for TO, which is based on DIOR (Decomposition-based Inner- and Outer-Refinement), a new algorithm for solving MIQQPs and MINLPs by successively refinining a nonconvex approximation. We are implementing the TO algorithm in Python/Pyomo within Decoco, a new decomposition-based MINLP-solver. Preliminary numerical results will be presented.
Ivo Nowak
added 8 research items
The purpose of this paper is threefold. First we show that the Lagrangian dual of a block-separable mixed-integer all-quadratic program (MIQQP) can be formulated as an eigenvalue optimization problem keeping the block-separable structure. Second we propose splitting schemes for reformulating non-separable problems as block-separable problems. Finally we report numerical results on solving the eigenvalue optimization problem by a proximal bundle algorithm applying Lagrangian decomposition. The results indicate that appropriate block-separable reformulations of MIQQPs could accelerate the running time of dual solution algorithms considerably.
Ivo Nowak
added 3 research items
The paper describes a software package called LaGO for solving nonconvex mixed integer nonlinear programs (MINLPs). The main component of LaGO is a convex relaxation which is used for generating solution candidates and computing lower bounds of the optimal value. The relaxation is generated by reformulating the given MINLP as a block-separable problem, and replacing nonconvex functions by convex underestimators. Results on medium size MINLPs are presented. AMS classifications: 90C22, 90C20, 90C27, 90C26, 90C59
International Series of Numerical Mathematics Relaxation and Decomposition Methods for Mixed Integer Nonlinear Programming 10.1007/3-7643-7374-1_14 IvoNowak 14.LaGO — An Object-Oriented Library for Solving MINLPs
We present a Branch and Cut algorithm of the software package LaGO to solve nonconvex mixed-integer nonlinear programs. A linear outer approximation is constructed from a convex relaxation of the problem. Since we do not require an algebraic representation of the problem, reformulation techniques for the construction of the convex relaxation cannot be applied, and we are restricted to sampling techniques in case of nonquadratic nonconvex functions. The linear relaxation is further improved by mixed-integer-rounding cuts. Also box reduction techniques are applied to improve efficiency. Numerical results on medium size testproblems and on the optimization of the design of an energy conversion system are presented to show the efficiency of the method.
Ivo Nowak
added an update
I added the slides of Dior, a new exact MINLP algorithm without branching, presented today at the OR 2017 in Berlin
 
Ivo Nowak
added a research item
Traditional deterministic global optimization methods are often based on a (branch-and-bound) search tree, which may grow rapidly, preventing the method to find a good solution. Motivated by decomposition-based inner approximation (column generation) methods for solving transport scheduling problems with over 100 million variables, we present a new deterministic decomposition-based successive approximation method for general modular and/or sparse global optimization problems, which is not based on a search tree. The new method, called Decomposition-based Inner- and Outer-Refinement (DIOR), is based on a block-separable reformulation of the model into sub-models, which can be solved in parallel. It generates inner- and outer-approximations using column generation, which are successively refined by solving nonconvex piecewise linear approximations. We present preliminary numerical results with Decogo (Decomposition-based Global Optimizer), a new parallel decomposition MINLP solver implemented in Python and Pyomo.
Ivo Nowak
added 3 research items
This book presents a comprehensive description of theory, algorithms and software for solving nonconvex mixed integer nonlinear programs (MINLP). The main focus is on deterministic global optimization methods, which play a very important role in integer linear programming, and are used only recently in MINLP. The presented material consists of two parts. The first part describes basic optimization tools, such as block-separable reformulations, convex and Lagrangian relaxations, decomposition methods and global optimality criteria. Some of these results are presented here for the first time. The second part is devoted to algorithms. Starting with a short overview on existing methods, deformation, rounding, partitioning and Lagrangian heuristics, and a branch-cut-and-price algorithm are presented. The algorithms are implemented as part of an object-oriented library, called LaGO. Numerical results on several mixed integer nonlinear programs are reported to show abilities and limits of the proposed solution methods. The book contains many illustrations and an up-to-date bibliography. Because of the emphasis on practical methods, as well as the introduction into the basic theory, it is accessible to a wide audience and can be used both as a research as well as a graduate text.
The purpose of this paper is threefold. First we propose splitting schemes for re- formulating non-separable problems as block-separable problems. Second we show that the Lagrangian dual of a block-separable mixed-integer all-quadratic program (MIQQP) can be formulated as an eigenvalue optimization problem keeping the block-separable structure. Fi- nally we report numerical results on solving the eigenvalue optimization problem by a proximal bundle algorithm applying Lagrangian decomposition. The results indicate that appropriate block-separable reformulations of MIQQPs could accelerate the running time of dual solution algorithms considerably.
Most industrial optimization problems are sparse, and can be reformulated by smaller sub-problems, which are linked by coupling constraints. Typical examples are planning, control and design problems. In practice, parallel decomposition methods are sometimes the only possibility to compute high-quality solutions of large-scale optimization problems. However, efficient implementations may require expert knowledge and problem-specific tricks. Recently, there is renewed interest in making these methods accessible to general users by developing generic decomposition frameworks and modeling support. These efforts are still early in the development stages, and there is much room for improvements. The purpose of this paper is twofold. On one hand we show how nonconvex airline planning and control problems with several hundred millions of variables and constraints can be solved in reasonable time by parallel decomposition methods. On the other hand, we present a novel decomposition approach for nonconvex programming with large duality gaps based on a nonconvex master (global approximation) problem. The method can be applied to general nonconvex optimization problems since an arbitrary sub-solver can be used. In particular, it can be applied to black-box simulation-based design optimization problems if a derivative free optimization method is used as a sub-problem solver.
Ivo Nowak
added an update
Project goal
The goals of this project are (i) development of new parallel decomposition methods for deterministic global optimization (ii) development of the new open-source MINLP-solver Decogo and (iii) solving difficult industrial optimization models using the new methods.
Background and motivation
Traditional deterministic global optimization methods are often based on a branch-and-bound tree, which may grow rapidly, preventing the method to find a good solution. Motivated by column generation methods for solving transport scheduling problems with over 100 million variables, we started to develop Decogo (Decomposition-based Global Optimizer), an open-source parallel MINLP solver implemented in Python and Pyomo.
Decogo can be applied to general modular and/or sparse optimization models.
It is based on a block-separable reformulation of the model into sub-models,
which can be solved in parallel. The main algorithms of Degogo are DIOR (Decomposition-based Inner- and Outer-Refinement) for deterministic global optimization and ADCG (Alternating Direction Column Generation Method) for computing local solutions.
We are planning to apply Decogo to solve complex energy system planning problems and engineering design problems.