Jan Kronqvist

Jan Kronqvist
KTH Royal Institute of Technology | KTH · Department of Mathematics (SCI-MAT)

Assistant Professor

About

56
Publications
17,510
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
990
Citations
Introduction
Jan Kronqvist is Assistant Professor at the the department of Mathematics at KTH in the division of Optimization and Systems Theory. Jan does research in Optimization (Mathematical Programming), Mixed-Integer Optimization, Integer optimization for AI and ML, and Process Systems Engineering

Publications

Publications (56)
Preprint
Full-text available
In this paper, we present a novel nonlinear programming-based approach to fine-tune pre-trained neural networks to improve robustness against adver-sarial attacks while maintaining high accuracy on clean data. Our method introduces adversary-correction constraints to ensure correct classification of adversarial data and minimizes changes to the mod...
Article
Full-text available
When faced with a limited budget of function evaluations, state-of-the-art black-box optimization (BBO) solvers struggle to obtain globally, or sometimes even locally, optimal solutions. In such cases, one may pursue solution polishing, i.e., a computational method to improve (or “polish”) an incumbent solution, typically via some sort of evolution...
Preprint
Full-text available
This paper introduces the Bi-linear consensus Alternating Direction Method of Multipliers (Bi-cADMM), aimed at solving large-scale regularized Sparse Machine Learning (SML) problems defined over a network of computational nodes. Mathematically, these are stated as minimization problems with convex local loss functions over a global decision vector,...
Article
Full-text available
In recent years, an interest appeared in integrating various optimization algorithms in machine learning. We study the potential of ensemble learning in classification tasks and how to efficiently decompose the underlying optimization problem. Ensemble learning has become popular for machine learning applications and it is particularly interesting...
Article
Full-text available
This paper describes a simple, but effective sampling method for optimizing and learning a discrete approximation (or surrogate) of a multi-dimensional function along a one-dimensional line segment of interest. The method does not rely on derivative information and the function to be learned can be a computationally-expensive “black box” function t...
Chapter
The presented work addresses two-stage stochastic programs (2SPs), a broadly applicable model to capture optimization problems subject to uncertain parameters with adjustable decision variables. In case the adjustable or second-stage variables contain discrete decisions, the corresponding 2SPs are known to be \(\textrm{NP}\)-complete. The standard...
Article
This paper presents a framework for computing the Gromov-Wasserstein problem between two sets of points in low dimensional spaces, where the discrepancy is the squared Euclidean norm.The Gromov-Wasserstein problem is a generalization of the optimal transport problem that finds the assignment between two sets preserving pairwise distances as much as...
Chapter
In this work, we develop a novel input feature selection framework for ReLU-based deep neural networks (DNNs), which builds upon a mixed-integer optimization approach. While the method is generally applicable to various classification tasks, we focus on finding input features for image classification for clarity of presentation. The idea is to use...
Article
Full-text available
In the present article we propose a mixed-integer approximation of adjustable-robust optimization problems, that have both, continuous and discrete variables on the lowest level. As these trilevel problems are notoriously hard to solve, we restrict ourselves to weakly-connected instances. Our approach allows us to approximate, and in some cases exa...
Preprint
This paper describes a simple, but effective sampling method for optimizing and learning a discrete approximation (or surrogate) of a multi-dimensional function along a one-dimensional line segment of interest. The method does not rely on derivative information and the function to be learned can be a computationally-expensive ``black box'' function...
Preprint
Full-text available
This paper presents a framework for computing the Gromov-Wasserstein problem between two sets of points in low dimensional spaces, where the discrepancy is the squared Euclidean norm. The Gromov-Wasserstein problem is a generalization of the optimal transport problem that finds the assignment between two sets preserving pairwise distances as much a...
Presentation
We present new decomposition methods for globally solving complex optimization and machine learning problems based on a generate-refine-and-solve (GRS) approach using inner and outer approximations. The methods are implemented in the open-source frameworks Decogo and Decolearn. Numerical results for complex nonconvex MINLPs are presented. Furthermo...
Preprint
Full-text available
0000−0003−0299−5745] , Boda Li 2[0000−0002−7934−9920] , Jan Rolfes 1[0000−0002−5415−1715] , and Shudian Zhao 1[0000−0001−6352−0968] Abstract. The presented work addresses two-stage stochastic programs (2SPs), a broadly applicable model to capture optimization problems subject to uncertain parameters with adjustable decision variables. In case the a...
Conference Paper
Full-text available
Currently minor improvements in image classification tasks are achieved by significantly increasing model complexity. This trend has been ongoing for the last years and high performance models now usually have millions of parameters. Inspired by Ensemble methods, we investigate the potential of Ensemble Learning (EL) to iteratively extend an ensemb...
Preprint
Full-text available
The number of cancer cases per year is rapidly increasing worldwide. In radiation therapy (RT), radiation from linear accelerators is used to kill malignant tumor cells. Scheduling patients for RT is difficult both due to the numerous medical and technical constraints, and because of the stochastic inflow of patients with different urgency levels....
Preprint
Full-text available
In the present article we propose a mixed-integer approximation of adjustable-robust optimization (ARO) problems, that have both, continuous and discrete variables on the lowest level. As these tri-level problems are notoriously hard to solve, we restrict ourselves to weakly-connected instances. Our approach allows us to approximate, and in some ca...
Preprint
Full-text available
In this work, we develop a novel input feature selection framework for ReLU-based deep neural networks (DNNs), which builds upon a mixed-integer optimization approach. While the method is generally applicable to various classification tasks, we focus on finding input features for image classification for clarity of presentation. The idea is to use...
Preprint
Full-text available
This paper proposes an open-source distributed solver for solving Sparse Convex Optimization (SCO) problems over computational networks. Motivated by past algorithmic advances in mixed-integer optimization, the Sparse Convex Optimization Toolkit (SCOT) adopts a mixed-integer approach to find exact solutions to SCO problems. In particular, SCOT brin...
Presentation
Full-text available
We present inner approximations methods for nonconvex MINLP and Ensemble Learning
Article
Full-text available
In this work, we extend the regularization framework from Kronqvist et al. (Math Program 180(1):285–310, 2020) by incorporating several new regularization functions and develop a regularized single-tree search method for solving convex mixed-integer nonlinear programming (MINLP) problems. We propose a set of regularization functions based on distan...
Article
Full-text available
In this paper, an open-source solver for mixed-integer nonlinear programming (MINLP) problems is presented. The Supporting Hyperplane Optimization Toolkit (SHOT) combines a dual strategy based on polyhedral outer approximations (POA) with primal heuristics. The POA is achieved by expressing the nonlinear feasible set of the MINLP problem with linea...
Preprint
Full-text available
We develop a class of mixed-integer formulations for disjunctive constraints intermediate to the big-M and convex hull formulations in terms of relaxation strength. The main idea is to capture the best of both the big-M and convex hull formulations: a computationally light formulation with a tight relaxation. The "$P$-split" formulations are based...
Article
It is well-documented how artificial intelligence can have (and already is having) a big impact on chemical engineering. But classical machine learning approaches may be weak for many chemical engineering applications. This review discusses how challenging data characteristics arise in chemical engineering applications. We identify four characteris...
Preprint
Full-text available
It is well-documented how artificial intelligence can have (and already is having) a big impact on chemical engineering. But classical machine learning approaches may be weak for many chemical engineering applications. This review discusses how challenging data characteristics arise in chemical engineering applications. We identify four characteris...
Article
The use of commercial flowsheeting programs enables straight-forward use of rigorous, but user hidden, mathematical formulations of chemical processes. The optimization of such black-box models is a challenging task due to nonconvexity, absence of accurate derivatives, and simulation convergence failures which can prevent classical optimization pro...
Conference Paper
Full-text available
This paper introduces a class of mixed-integer formulations for trained ReLU neural networks. The approach balances model size and tightness by partitioning node inputs into a number of groups and forming the convex hull over the partitions via disjunctive programming. At one extreme, one partition per input recovers the convex hull of a node, i.e....
Article
Full-text available
Generating polyhedral outer approximations and solving mixed-integer linear relaxations remains one of the main approaches for solving convex mixed-integer nonlinear programming (MINLP) problems. There are several algorithms based on this concept, and the efficiency is greatly affected by the tightness of the outer approximation. In this paper, we...
Chapter
This work develops a class of relaxations in between the big-M and convex hull formulations of disjunctions, drawing advantages from both. The proposed “P-split” formulations split convex additively separable constraints into P partitions and form the convex hull of the partitioned disjuncts. Parameter P represents the trade-off of model size vs. r...
Article
Gradient boosted trees and other regression tree models perform well in a wide range of real-world, industrial applications. These tree models (i) offer insight into important prediction features, (ii) effectively manage sparse data, and (iii) have excellent prediction capabilities. Despite their advantages, they are generally unpopular for decisio...
Article
Full-text available
Different versions of polyhedral outer approximation are used by many algorithms for mixed-integer nonlinear programming (MINLP). While it has been demonstrated that such methods work well for convex MINLP, extending them to solve nonconvex problems has traditionally been challenging. The Supporting Hyperplane Optimization Toolkit (SHOT) is a solve...
Preprint
Full-text available
This paper introduces a class of mixed-integer formulations for trained ReLU neural networks. The approach balances model size and tightness by partitioning node inputs into a number of groups and forming the convex hull over the partitions via disjunctive programming. At one extreme, one partition per input recovers the convex hull of a node, i.e....
Preprint
Full-text available
This work develops a class of relaxations in between the big-M and convex hull formulations of disjunctions, drawing advantages from both. The proposed "P-split" formulations split convex additively separable constraints into P partitions and form the convex hull of the partitioned disjuncts. Parameter P represents the trade-off of model size vs. r...
Preprint
Full-text available
In this paper, a recently released open-source solver for convex mixed-integer nonlinear programming (MINLP) is presented. The Supporting Hyperplane Optimization Toolkit (SHOT) solver combines a dual strategy based on polyhedral outer approximations (POA) with several primal heuristics. The outer approximation is achieved by expressing the nonlinea...
Article
Full-text available
We introduce an efficient method for the verification of ReLU-based feed-forward neural networks. We derive an automated procedure that exploits dependency relations between the ReLU nodes, thereby pruning the search tree that needs to be considered by MILP-based formulations of the verification problem. We augment the resulting algorithm with meth...
Preprint
Full-text available
Different versions of polyhedral outer approximation is used by many algorithms for mixed-integer nonlinear programming (MINLP). While it has been demonstrated that such methods work well for convex MINLP, extending them to solve also nonconvex problems has been challenging. One solver based on outer linearization of the nonlinear feasible set of M...
Preprint
Full-text available
Gradient boosted trees and other regression tree models perform well in a wide range of real-world, industrial applications. These tree models (i) offer insight into important prediction features, (ii) effectively manage sparse data, and (iii) have excellent prediction capabilities. Despite their advantages, they are generally unpopular for decisio...
Conference Paper
We introduce an efficient method for the verification of ReLU-based feed-forward neural networks. We derive an automated procedure that exploits dependency relations between the ReLU nodes, thereby pruning the search tree that needs to be considered by MILP-based formulations of the verification problem. We augment the resulting algorithm with meth...
Chapter
Full-text available
The Supporting Hyperplane Optimization Toolkit (SHOT) solver was originally developed for solving convex MINLP problems, for which it has proven to be very efficient. In this paper, we describe some techniques and strategies implemented in SHOT for improving its performance on nonconvex problems. These include utilizing an objective cut to force an...
Chapter
Gradient boosted trees and other regression tree models are known to perform well in a wide range of real-world, industrial applications. These tree models (i) offer insight into important prediction features, (ii) effectively manage sparse data, and (iii) have excellent prediction capabilities. We consider holistic decision-making problems where p...
Article
Full-text available
In this paper, we present a review of deterministic software for solving convex MINLP problems as well as a comprehensive comparison of a large selection of commonly available solvers. As a test set, we have used all MINLP instances classified as convex in the problem library MINLPLib, resulting in a test set of 335 convex MINLP instances. A summar...
Conference Paper
Full-text available
In this paper, it is explained how algorithms for convex mixed-integer nonlinear programming (MINLP) based on poly-hedral outer approximation (POA) can be integrated with mixed-integer programming (MIP) solvers through callbacks and lazy constraints. Through this integration, a new approach utilizing a single branching tree is obtained which reduce...
Article
In this paper, we present two new methods for solving convex mixed-integer nonlinear programming problems based on the outer approximation method. The first method is inspired by the level method and uses a regularization technique to reduce the step size when choosing new integer combinations. The second method combines ideas from both the level m...
Thesis
Full-text available
This thesis is focused on a specific type of optimization problems commonly referred to as convex MINLP problems. The goal has been to investigate and develop efficient methods for solving such optimization problems. The thesis focuses on decomposition-based algorithms in which a polyhedral outer approximation is used for approximating the integer...
Article
Full-text available
Several deterministic methods for convex mixed integer nonlinear programming generate a polyhedral approximation of the feasible region, and utilize this approximation to obtain trial solutions. Such methods are, e.g., outer approximation, the extended cutting plane method and the extended supporting hyperplane method. In order to obtain the optima...
Article
Here we present a center-cut algorithm for convex mixed-integer nonlinear programming (MINLP) that can either be used as a primal heuristic or as a deterministic solution technique. Like several other algorithms for convex MINLP, the center-cut algorithm constructs a linear approximation of the original problem. The main idea of the algorithm is to...
Chapter
In this work, we study the performance of the mixed-integer nonlinear programming solver extended cutting plane (ECP) for solving simulation-based optimization problems. This solver, unlike other commercial derivative-based solvers such as the ones based on the branch and bound or outer approximation algorithms, does not require solving a nonlinear...
Article
Full-text available
In this paper, we generalize the extended supporting hyperplane algorithm for a convex continuously differentiable mixed-integer nonlinear programming problem to solve a wider class of nonsmooth problems. The generalization is made by using the subgradients of the Clarke subdifferential instead of gradients. Consequently, all the functions in the p...
Article
Full-text available
In this paper, the problem of simultaneously estimating the structure and parameters of artificial neural networks with multiple hidden layers is considered. A method based on sparse optimization is proposed. The problem is formulated as an ℓ0-norm minimization problem, so that redundant weights are eliminated from the neural network. Such problems...
Chapter
In this paper a new open source solver for convex mixed-integer nonlinear programming (MINLP) implemented in Wolfram Mathematica is described. The Supporting Hyperplane Optimization Toolkit (SHOT) solver implements two methods for MINLP based on polyhedral outer approximations, namely the Extended Supporting Hyperplane (ESH) and Extended Cutting Pl...
Chapter
In this paper, we present a new algorithm for solving convex mixed-integer nonlinear programming problems. Similarly to other linearization-based methods, the algorithm generates a polyhedral approximation of the feasible region. The main idea behind the algorithm is to use a different approach for obtaining trial solutions. Here trial solutions ar...
Article
A new deterministic algorithm for solving convex mixed-integer nonlinear programming (MINLP) problems is presented in this paper: The extended supporting hyperplane (ESH) algorithm uses supporting hyperplanes to generate a tight overestimated polyhedral set of the feasible set defined by linear and nonlinear constraints. A sequence of linear or qua...

Network

Cited By