Algorithm 856: APPSPACK 4.0: asynchronous parallel pattern search for derivativefree optimization.
ABSTRACT APPSPACK is software for solving unconstrained and boundconstrained optimization problems. It implements an asynchronous parallel pattern search method that has been specifically designed for problems characterized by expensive function evaluations. Using APPSPACK to solve optimization problems has several advantages: No derivative information is needed; the procedure for evaluating the objective function can be executed via a separate program or script; the code can be run serially or in parallel, regardless of whether the function evaluation itself is parallel; and the software is freely available. We describe the underlying algorithm, data structures, and features of APPSPACK version 4.0, as well as how to use and customize the software.
 [Show abstract] [Hide abstract]
ABSTRACT: Optimization is often used to perform model calibration, the process of inferring the values of model parameters so that the results of the simulations best match observed behavior. It can both improve the predictive capability of the model and curtail the loss of information caused by using a numerical model instead of the actual system. At its heart is the comparison of experimental data and simulation results. Complicating this comparison is the fact that both data sets contain uncertainties which must be quantified in order to make reasonable comparisons. Therefore, uncertainty quantification (UQ) techniques can be applied to identify, characterize, reduce, and, if possible, eliminate uncertainties. Incorporation of UQ into the calibration process can drastically improve the usefulness of computational models. Current approaches are serial approaches in that first, the calibration parameters are identified and then, a series of runs dedicated to UQ analysis is completed. Although this approach can be effective, it can be computationally expensive or produce incomplete results. Model analysis that takes advantage of intermediate optimization iterates can reduce the expense, but the sampling done by the optimization algorithms is not ideal. In this paper, we will review serial approaches and propose a joint calibration and UQ approach that combines Bayesian statistical models and derivativefree optimization in order to monitor sensitivity information throughout the calibration process.Journal of Computational Methods in Sciences and Engineering 01/2012; 12(12):99110.  SourceAvailable from: Herbert K. H. Lee[Show abstract] [Hide abstract]
ABSTRACT: This work combines pattern search optimization with a statistical emulator based on Treed Gaussian Processes (TGP) to create a new hybrid algorithm. The goal is to use the global probabilistic view provided by TGP to inform the local pattern search and form a more intelligent optimization algorithm. We also propose ways in which the emulator can be used to gain information about the objective function, inform the algorithm stopping rules and provide a probabilistic analysis of the type of convergence. We present the algorithm, a framework for statistically informed optimization, and illustrate the work with numerical results.  [Show abstract] [Hide abstract]
ABSTRACT: Modern IC power delivery systems encompass large onchip passive power grids and active onchip or offchip voltage converters and regulators. While there exists little work targeting on holistic design of such complex IC subsystems, the optimal systemlevel design of power delivery is critical for achieving power integrity and power efficiency. In this article, we conduct a systematic design analysis on power delivery networks that incorporate Buck Converters (BCs) and onchip LowDropout voltage regulators (LDOs) for the entire chip power supply. The electrical interactions between active voltage converters, regulators as well as passive power grids and their influence on key system design specifications are analyzed comprehensively. With the derived design insights, the systemlevel codesign of a complete power delivery network is facilitated by a proposed automatic optimization flow in which key design parameters of buck converters and onchip LDOs as well as onchip decoupling capacitance are jointly optimized. The experimental results demonstrate significant performance improvements resulted from the proposed system cooptimization in terms of achievable area overhead, supply noise and power efficiency. Impacts of different decoupling capacitance technologies are also investigated.ACM Transactions on Design Automation of Electronic Systems (TODAES). 03/2013; 18(2).
Page 1
SANDIA REPORT
SAND20046391
Unlimited Release
Printed December 2004
APPSPACK 4.0: Asynchronous
Parallel Pattern Search for
DerivativeFree Optimization
Genatha A. Gray and Tamara G. Kolda
Prepared by
Sandia National Laboratories
Albuquerque, New Mexico 87185 and Livermore, California 94550
Sandia is a multiprogram laboratory operated by Sandia Corporation,
a Lockheed Martin Company, for the United States Department of Energy’s
National Nuclear Security Administration under Contract DEAC0494AL85000.
Approved for public release; further dissemination unlimited.
Page 2
Issued by Sandia National Laboratories, operated for the United States Department of
Energy by Sandia Corporation.
NOTICE: This report was prepared as an account of work sponsored by an agency of
the United States Government. Neither the United States Government, nor any agency
thereof, nor any of their employees, nor any of their contractors, subcontractors, or their
employees, make any warranty, express or implied, or assume any legal liability or re
sponsibility for the accuracy, completeness, or usefulness of any information, appara
tus, product, or process disclosed, or represent that its use would not infringe privately
owned rights. Reference herein to any specific commercial product, process, or service
by trade name, trademark, manufacturer, or otherwise, does not necessarily constitute
or imply its endorsement, recommendation, or favoring by the United States Govern
ment, any agency thereof, or any of their contractors or subcontractors. The views and
opinions expressed herein do not necessarily state or reflect those of the United States
Government, any agency thereof, or any of their contractors.
Printed in the United States of America. This report has been reproduced directly from
the best available copy.
Available to DOE and DOE contractors from
U.S. Department of Energy
Office of Scientific and Technical Information
P.O. Box 62
Oak Ridge, TN 37831
Telephone:
Facsimile:
EMail:
Online ordering:
(865) 5768401
(865) 5765728
reports@adonis.osti.gov
http://www.doe.gov/bridge
Available to the public from
U.S. Department of Commerce
National Technical Information Service
5285 Port Royal Rd
Springfield, VA 22161
Telephone:
Facsimile:
EMail:
Online ordering:
(800) 5536847
(703) 6056900
orders@ntis.fedworld.gov
http://www.ntis.gov/ordering.htm
Page 3
SAND20046391
Unlimited Release
Printed December 2004
APPSPACK 4.0: Asynchronous Parallel Pattern
Search for DerivativeFree Optimization
Genetha A. Gray and Tamara G. Kolda
Computational Sciences and Mathematics Research Department
Sandia National Laboratories
Livermore, CA 94551–9217
{gagray,tgkolda}@sandia.gov
Abstract
APPSPACK is software for solving unconstrained and bound constrained optimization prob
lems. It implements an asynchronous parallel pattern search method that has been specifically
designed for problems characterized by expensive function evaluations. Using APPSPACK to
solve optimization problems has several advantages: No derivative information is needed; the
procedure for evaluating the objective function can be executed via a separate program or script;
the code can be run in serial or parallel, regardless of whether or not the function evaluation
itself is parallel; and the software is freely available. We describe the underlying algorithm, data
structures, and features of APPSPACK version 4.0 as well as how to use and customize the
software.
3
Page 4
Contents
1 Introduction.................................................................5
2 APPS algorithm & implementation ..........................................
2.1Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2 Constraints and Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3Search Directions and Step Lengths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.4Generation of Trial Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.5Stopping Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.6The Evaluation Conveyor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.7Executors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.8Evaluators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.9Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6
8
9
10
10
10
11
13
14
17
3 Using APPSPACK ..........................................................
3.1Download, Compiling, and Installing APPSPACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2Creating a Function Evaluation for APPSPACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3Creating the APPSPACK Input File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4Running APPSPACK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5APPSPACK Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.6User Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
19
20
21
24
24
26
4 Customizing APPSPACK.................................................... 27
4.1Customizing the Function Evaluations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.2Customizing the Parallelization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3General Linear and Nonlinear Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
27
28
5Conclusions.................................................................. 29
References...................................................................... 32
Figures
1
2
3
4
5
6
7
8
9
APPS Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Determining if point y is “better” than point x (y ≺ x). . . . . . . . . . . . . . . . . . . . . . . . .
Conveyor actions for trial point exchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Calling APPSPACK in Serial Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Calling APPSPACK in MPI Mode (Manager) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
APPSPACK MPI Worker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The “system call” evaluator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Parallel APPSPACK using the ”system call” evaluator.. . . . . . . . . . . . . . . . . . . . . . . . .
Parallel APPSPACK using a customized evaluator. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
8
12
14
15
16
17
18
28
4
Page 5
APPSPACK 4.0: Asynchronous
Parallel Pattern Search for
DerivativeFree Optimization
1 Introduction
APPSPACK is software for solving unconstrained and boundconstrained optimization problems,
i.e., problems of the form
min
subject to
f(x)
l ≤ x ≤ u.
(1)
Here, f : Rn→ R∪{+∞} and x ∈ Rn. The upper and lower bounds are optional on an elementby
element basis; specifically, l is an nvector with entries in R∪{−∞} and u is an nvector with entries
in R∪{+∞}. To find a solution of (1), APPSPACK implements asynchronous parallel pattern search
(APPS) [Hough et al. 2001; Kolda 2004], a method in the class of direct search methods [Wright
1996; Lewis et al. 2000]. APPS is a variant on generating set search as described by Kolda et al.
[2003]. Since it is a direct search method, APPS does not require any gradient information and is
thus applicable to a variety of contexts. APPS is provably convergent if the underlying objective
function is suitably smooth [Kolda and Torczon 2003; Kolda and Torczon 2004; Kolda 2004].
APPSPACK is targeted to simulationbased optimization. These problems are characterized by
a relatively small number of variables (i.e., n < 100), and an objective function whose evaluation
requires the results of a complex simulation. One standard application of this kind of optimization
is parameter estimation. The goal in this case is to identify the set of simulator input parameters
that produces output that most closely matches some given observed data. For this problem, the
objective function might be of the form
f(x) =
N
?
i=1
(si(x) − oi)2.
Here, N denotes the number of data points to be compared; for example, the points could correspond
to times or spatial locations. The values oifor i = 1,...,N are the given observed data values at
these points, and the values si(x) for i = 1,...,N are the simulator outputs at the same points,
depending on the input x. Note that in order to discover the x that yields the best fit to the observed
data, multiple simulations are required.
Using APPSPACK to solve optimization problems has the following advantages:
• No derivative information in needed.
• The procedure for evaluating the objective function does not need to be encapsulated in a
subroutine and can, in fact, be an entirely separate program.
• The code can be run in serial or in parallel, regardless of whether or not the objective function
itself runs in parallel.
• The software is freely available under the terms of the LGPL.
These advantages have prompted users to employ APPSPACK for a wide variety of applications.
See, for example, Hough et al. [2001], Mathew et al. [2002], Chiesa et al. [2004],Kupinksi et al. [2003],
Croue [2003], Gray et al. [2003], and Fowler et al. [2004].
5
Page 6
APPSPACK version 4.0 is based on the specific algorithm described by Kolda [2004]. It is written
in C++ and uses MPI [Gropp et al. 1996; Gropp and Lusk 1996] for parallelism. The underlying
algorithm and its class structures are described in Section 2. Potential users of APPSPACK will be
most interested in Section 3, where we explain how to obtain, compile, install, and use APPSPACK,
and in Section 5 where we summarize some successful applications. In addition, customizations
of APPSPACK are discussed in Section 4. Sections 4.1 and 4.2 describe how the software can be
directly linked with the objective function and how different means of communication can be used
for the function evaluation. An ad hoc means of handling of general linear and nonlinear constraints
is also discussed (see Section 4.3). Implementation of theoretically more robust ways to handle these
types of constraints is underway and will be communicated in a future publication and software
release.
The notation in this paper is as follows. A boldface capital letter, e.g., T, denotes a set of vectors.
A script capital letter, e.g., I, denotes a set of indices. A boldface lowercase letter, e.g., x, denotes
a vector, and its ith entry is denoted by xi. Note that direpresents the ith vector in a set of vectors
and not the ith component in d, which would instead be denoted di(no boldface). Greek letters,
e.g., α, represent scalars.
2 APPS algorithm & implementation
The APPS algorithm includes numerous details that are essential for efficient and correct implemen
tation but not for basic understanding of the method. Omitting these details, APPS can be simply
described as follows.
1. Generate a set of trial points to be evaluated,
T = {x + ∆idi: i ∈ I}.
(2)
Here, x is the best point known so far, diis the ith search direction, ∆iis the corresponding step
length, and I is the subset of search directions for which new trial points should be generated.
2. Send the set T to the conveyor for evaluation, and collect a set of evaluated points, E, from
the conveyor. (The conveyor is a mechanism for shuttling trial points through the process of
being evaluated.)
3. Process the set E to see if any point in E is “better than” x. In other words, check if there
exists a y ∈ E such that
y ≺ x,
where the notation “≺” denotes the “better than” relationship which essentially means that
the point y has a lower function value. (It is described in detail in Figure 2.) If E contains a
point better than x, then the iteration is successful; otherwise, it is unsuccessful.
(3)
4. If the iteration is successful, replace x with the new best point (from E). Optionally, regenerate
the set of search directions and delete any pending trial points in the conveyor.
5. If the iteration is unsuccessful, reduce certain step lengths as appropriate. In addition, check
for convergence based on the step lengths.
A detailed procedural version of APPS is given in Figure 1; for a complete mathematical de
scription and analysis, see Kolda [2004]. The remainder of this section describes the implementation
details and specific C++ objects of APPSPACK version 4.0.
6
Page 7
Initialization.
Choose initial guess x ∈ Rnand tolerances ∆minand ∆tol.
Let D = {d1,...,dp} be a set of search directions that conform to the nearby boundary.
Set ∆i= ∆initand τi= −1 for i = 1,...,p.
Iteration.
(1) Generate Trial Points.
Let I := {i : ∆i≥ ∆toland τi= −1}. For each i ∈ I:
– Let˜∆i≤ ∆ibe the length of the longest feasible step from x along direction di.
– Create a new trial point y := x +˜∆idiand an associated tag.
– Along with y, save the following information.
◦ Parent Tag(y) := Tag(x)
◦ Direction Index(y) := i
◦ Step(y) := ∆i
– Set τi:= Tag(y).
– Add the new trial point to the set T.
(2) Exchange Trial Points.
Send the (possibly empty) set of new trial points T to the evaluation conveyor.
Collect a set E of trial points that have been evaluated.
(3) Process Evaluated Trial Points.
Let z denote the “best” point in E.
If z is “better than” x, goto Step 4; otherwise, goto Step 5.
(4) Successful Step.
– Delete x, remove z from E, and reset x := z. Delete remaining points in E.
– Check for convergence based on the function value.
– Compute search directions D = {d1,...,dp} that conform to the nearby boundary (p
may also change).
– Set ∆i:= max{Step(z),∆min} and τi:= −1 for i = 1,...,p.
– Prune the evaluation queue.
– Go to Step 1.
(5) Unsuccessful Step.
– For each y ∈ E: If Parent Tag(y) = Tag(x), then let i = Direction Index(y)
and set ∆i:=1
– Delete all points in E.
– Check for convergence based on the step lengths.
– Go to Step 1.
2∆iand τi:= −1.
Figure 1. APPS Algorithm
7
Page 8
2.1Points
Points (i.e., the best point and trial points) are stored as APPSPACK::Point objects. As previously
described, new points are generated according to equation (2), and each new point y is of the form
y = x + ∆idi,
(4)
where x is the parent, ∆iis the step length, and diis the direction. Besides the vector y ∈ Rnitself,
each Point object stores some additional relevant information. Every Point includes a unique tag,
a positive integer that is used as an identifier. In addition, each Point contains information about
the parent (i.e., the vector x), search direction, and step length used to generate y according to
equation (4). Once a trial point has been evaluated, its function value, f(y) ∈ R ∪ {+∞}, is also
stored in Point. This function value is stored as a APPSPACK::Value object which was created to
handle the possibility that f(y) = +∞. APPSPACK uses the special case f(y) = +∞ to signify
trial points that could not be evaluated (e.g., the simulator failed) and certain types of infeasible
points.
In order for a trial point y to be the next best point, it must satisfy two conditions. First, it must
satisfy a decrease condition with respect to its parent, and second, it must have a function value
that improves upon that of the current best point. We describe both of these conditions below.
A trial point y satisfies a decrease condition with respect to its parent x and step length ∆ if
the following holds:
f(y) < f(x) − α∆2,
where α ≥ 0 is a userdefined constant. If α = 0, this is called simple decrease [Torczon 1995];
otherwise, if α > 0, this is called sufficient decrease [Yu 1979; Lucidi and Sciandrone 2002]. To
indicate whether or not a trial point satisfies (5) with respect to its parent, state information is
stored in object Point. The state also specifies whether or not a trial point has been evaluated.
(5)
Satisfying decrease condition (5) is only part of the comparison APPSPACK uses to determine
whether or not one point is “better than” another. The comparison also considers the corresponding
function values and defines a scheme for tiebreaking in the case that these function values are equal.
The complete procedure for determining if a point y is better than a point x (i.e. whether y ≺ x)
is detailed in Figure 2.
Let y and x be Point objects. The following procedure determines if y ≺ x.
– If Tag(y) = Tag(x) (i.e., they are the same point), then y ?≺ x. Else, continue.
– If y does not satisfy (5) with respect to its parent, then y ?≺ x. Else, continue.
– If x does not satisfy (5) with respect to its parent, then y ≺ x. (Note that y has been
evaluated and does satisfy (5) by the previous bullet.) Else, continue.
– If f(y) < f(x), then y ≺ x. Else, continue.
– If f(y) > f(x), then y ?≺ x. Else, continue.
– It must be the case that f(y) = f(x), so we break the tie by choosing the point with the
lower tag. If Tag(y) < Tag(x), then y ≺ x. Else y ?≺ x.
Figure 2. Determining if point y is “better” than point x (y ≺ x).
8
Page 9
In summary, an APPSPACK Point object stores the following information:
• the vector y ∈ Rn;
• its unique tag, denoted Tag(y);
• its parent’s tag, denoted Parent Tag(y);
• its parent’s function value;
• the step used to produce it, denoted Step(y);
• the index of the direction used to produce it, denoted Direction Index(y).
• the state information;
• its function value f(y) (if it has been evaluated).
The state information indicates whether or not the point has been evaluated and, if it has, whether or
not it satisfies the sufficient decrease condition. Note that the parent vector is not explicitly stored;
instead, only its corresponding tag and function value are stored. Likewise, the actual direction
vector diis not stored; instead, only its index i is stored.
2.2Constraints and Scaling
Various types of constraints can potentially be handled by APPSPACK using the abstract interface
defined by the APPSPACK::Constraints::Interface class. Currently, we support only bound con
straints via the APPSPACK::Constraints::Bounds class, but have plans to support general linear
and nonlinear constraints using alternative constraint classes in the future. The constraints object is
passed as an argument to the APPSPACK::Solver. We employ this programming structure to leave
open the possibility of providing userdeveloped constraint classes in future versions.
The bounds on the variables are specified by the user in the APPSPACK input file (see Sec
tion 3.3). They are used both to determine a conforming set of search directions (see Section 2.3)
and to generate trial points (see Section 2.4).
Related to the bounds and stored in the same object is the variable scaling. Because derivative
free methods do not use any gradient information, proper scaling of the variables is critical. Al
though scaling is not explicitly mentioned in the description of APPS provided in Figure 1, it plays
important roles in convergence (see Section 2.5), determining a conforming set of search directions
(see Section 2.3), calculating trial points (see Section 2.4), and looking up points in the cache (see
Section 2.9).
To define a default scaling vector, APPSPACK uses the bounds. Specifically, let l,u ∈ Rndenote
the vectors of the lower and upper bounds, respectively. Then, the components of the scaling vector
s ∈ Rnare defined as
si= ui− lifor i = 1,...,n.
In the case that all the bounds are finite, the user may choose either to use this default scaling vector
or to provide one in the APPSPACK input file (“Scaling” in the “Bounds” sublist). If finite bounds
are not provided or do not exist, s must be provided by the user. Note that this approach to scaling
was motivated by another derivativefree optimization software package, IFFCO Choi et al. [1999],
a Fortran implementation of the implicit filtering method Gilmore and Kelley [1995; Kelley [1999].
(6)
9
Page 10
2.3Search Directions and Step Lengths
The search directions are handled by the APPSPACK::Directions class. A new set of search direc
tions, or a search pattern, is computed every time an iteration is successful (see Step 4 in Figure 1).
To generate this search pattern, APPSPACK considers the scaled coordinate directions and excludes
directions outside the tangent cone. Specifically, the set of search directions is defined as
D = {d1,...,dp} = {siei: xi< ui} ∪ {−siei: xi> ?i},
where diis the ith search direction and eiis the ith unit vector.
Each direction dihas a tag, τi, and step length, ∆i, associated with it. The tag is an integer
that indicates whether or not there are any points in the evaluation conveyor associated with a given
search direction. If τi= −1, then there are currently no unevaluated trial points with parent x that
were generated using direction di. In this case, the step length ∆iis the value that will be used in
Step 1 of Figure 1 to compute a new trial point in direction i. Otherwise, τiis the tag number of the
point associated with direction di, and ∆iis the step length that was used to generate that point.
2.4 Generation of Trial Points
Trial points are generated in the APPSPACK::Solver class. As indicated in Step 1 of Figure 1, a
trial point is computed for each direction i ∈ I where I = {i : ∆i≥ ∆toland τi= −1}. In other
words, the set I contains the search directions that have not yet converged (see Section 2.5) and do
not currently have a trial point in the evaluation conveyor.
For each i ∈ I, a feasible trial point is calculated. If y = x + ∆idi is not feasible, then an
appropriate pseudostep must be determined. The pseudo step,˜∆i, is the longest possible step that
is feasible, and it is formally defined as
˜∆i= max{∆ ∈ [0,∆i] : l ≤ x + ∆di≤ u}.
2.5Stopping Conditions
The primary stopping condition in the APPS algorithm is based on the step length. This criteria was
chosen because it can be shown that if the objective function is suitably smooth, then the gradient
can be bounded as multiple of the step size, i.e., ?∇f(x)? = O(maxi{∆i}) [Kolda et al. 2003]. In
other words, the steps only get smaller if the norm of the gradient is decreasing. Hence, the step
length can be used to define a stopping requirement.
The stopping condition based on step length is used in Step 5 of Figure 1. Specifically, APPS
converges if all the step lengths are less than the specified tolerance; i.e.,
∆i< ∆tolfor i = 1,...,p.
(7)
Here, we say that the ith direction is converged if ∆i< ∆tol. The tolerance ∆tolcan be specified by
the user in the APPSPACK input file (“Step Tolerance” in the “Solver” sublist). The default value
is ∆tol= 0.01 which corresponds to a 1% change in the variables when the default scaling defined
in (6) is used.
An alternative stopping condition is based on whether or not the function has reached a specified
threshold. This criteria may be useful when the desired minimum value is known. For example, in the
parameter estimation problem described in Section 1, it may be reasonable to stop when f(x) < 0.03
or when the fit is within 3% of being exact. Step 4 of Figure 1 shows the implementation of the
10
Page 11
function tolerance stopping criteria. Specifically, the iterations are terminated if
f(x) ≤ ftol,
(8)
where ftolis defined by the user in the APPSPACK input file (“Function Tolerance” in the “Solver”
sublist). By default, this stopping condition is not employed.
Stopping can also be defined in terms of the number of function evaluations. In other words,
the algorithm can be discontinued after a specified number of function evaluations has been com
pleted. This sort of stopping criteria might be useful when the function evaluations are based on a
particularly expensive simulation, and the user wants to adhere to a specified budget of evaluations.
By default, this stopping criteria is not used, but it can be activated by the user by specifying a
maximum number of function evaluations in the APPSPACK input file (“Maximum Evaluations”
in the “Solver” sublist).
2.6The Evaluation Conveyor
From the point of view of the APPS algorithm, the evaluation conveyor simply works as follows:
A set of unevaluated points T is exchanged for a set of evaluated points E (Step 2 of Figure 1).
The input set of unevaluated points may be empty. However, because returning an empty set of
evaluated points means that the current iteration cannot proceed, the output set of evaluated points
must always be nonempty.
Within the conveyor, a trial points moves through three stages. The first stage is to wait in a
holding pen (the “Wait” queue) until resources become available for evaluating its function value.
The second stage occurs while the function value is in the process of being computed, (in which case
it sits in the “Pending” queue). The third stage takes place after the evaluation has been completed,
while the trial point waits to be returned as output from the conveyor (in the “Return” queue).
Each of these stages is described in more detail below.
One key point in this process is that it may take more than one iteration for a point to move
through the conveyor. Thus, the set of points T that is input is not necessarily the same as the set
of points E that is output. Furthermore, because it may take multiple iterations for a point to move
through the conveyor, it is often desirable to remove some or all of the points that are in the first
stage of the conveyor, waiting for evaluation. This removal of points is called pruning.
Every point that is submitted to the evaluation conveyor is eventually either returned or pruned.
Pruning occurs in Step 4 of Figure 1 while returning takes place in Step 2. The evaluation conveyor
facilitates the movement of points through the following three queues:
• W, the “Wait” queue where trial points wait to be evaluated. This is the only queue from
which points can be pruned.
• P, the “Pending” queue for points with ongoing evaluations. Its size is restricted by the
resources available for function evaluations.
• R, the “Return” queue where evaluated points are collected. Its size can be controlled by the
user.
This conveyor process is handled by the APPSPACK::Conveyor object. Each time a set of trial points
is received (Step 2 of Figure 1), the conveyor follows the procedure diagrammed in Figure 3.
Points that are waiting to be submitted to the executor for evaluation are stored in W, and they
remain there until either there is space in P or they are pruned. The W queue is pruned whenever
an iteration is successful (Step 4 of Figure 1). By default, pruning is defined as the emptying of W.
11
Page 12
Set R = ∅.
Push new trial points onto W.
?
H
H
???? ?
HHHH H?
?
?
No
?
?
?
H
H
H
H
H
R ≥ MinReturn?
Yes
?
?
?
?
?
Return R
???? ?
HHHH H?
?
?
No
?
?
?
H
H
H
W and P empty?
Yes
6
?
H
H
???? ?
HHHH H
?
?
?
No
?
?
?
H
H
H
H
H
W empty?
Yes
?
No
???? ?
HHHH H
?
?
?
Yes
?
?
?
H
H
H
P < MaxPending?
????? ?
HHHH H
?
?
?
Yes
?
?
H
H
H
H
H
Evaluation
finished?
No

6
Pop evaluated point y from P
and push y with f(y) onto R
6
???? ?
HHHH H?
?
?
?
?
H
H
H
H
H
R < MaxReturn
No
Yes
?
6
Pop point y from W
?
H
H
???? ?
HHHH H
?
?
?
No
?
?
?
H
H
H
Is f(y) cached?
Yes

Push y with
f(y) onto R.

Push y onto P
(starting evaluation)
Figure 3. Conveyor actions for trial point exchange
12
Page 13
However, the results of a pruning can be modified by setting the “Max Queue Size” parameter in the
“Solver” sublist of the APPSPACK input file. Here, the user can specify the number of points that
should remain in W after it is pruned. In this case, the oldest points are deleted and the newest
points remain in the queue.
Before a point moves from W to P, the cache is checked to see if the function value has already
been calculated for that point (see Section 2.9). If so, the cached function value is obtained, and the
point is moved directly to R. If not, the point moves to P and is evaluated. Once a point has been
pushed from W onto P or R, it can no longer be pruned.
Points which have been submitted to the executor for evaluation are stored in P. The executor
handles the distribution of evaluation tasks to workers and is described in Section 2.7. The size of
P is solely determined by the executor and depends on available resources. Recall that APPSPACK
was designed to accommodate problems that may have expensive function evaluations.
points may remain in P for several iterations. Once the executor returns the results of the function
evaluation, the point is moved to R.
Hence,
Essentially, the conveyor process continues until enough evaluated points are collected in the
R. Enough is defined by the “Minimum Exchange Return” value set in the “Solver” sublist of
the APPSPACK input file. The default value is one, but larger values can be used to force the
conveyor to collect more evaluated trial points before returning.
process can continue until every trial point has been evaluated and collected. This behavior defines
a synchronous pattern search [Lewis and Torczon 1996] and can be activated by setting the parameter
“Synchronous” to true in the “Solver” sublist. (The default is false.) Finally, note that the size of
R can also be controlled by defining a “Maximum Exchange Return” size in the “Solver” sublist,
which defaults to 1000.
In the extreme, the conveyor
2.7Executors
When a point enters the second stage of the conveyor, the P queue, it must be assigned to a worker
(if running in parallel) and evaluated. The executor coordinates the assignment of points to workers
for function evaluation. Its pure virtual abstract interface is defined in APPSPACK::Executor::
Interface and includes the following:
• A boolean function that returns true if the executor is waiting to spawn more function eval
uations. The result of this function is used in the conveyor test P < MaxPending shown in
Figure 3.
• A spawn function that initiates a function evaluation. The input is a vector y and its cor
responding tag, Tag(y). The tag is used as an input argument in the function evaluation
executable, and it is needed to match the resulting function value with the appropriate point
in queue P.
• A receive function that checks whether or not any function evaluations have finished. If an
evaluation has been completed, the output is the objective function value and any related
information.
The executor is passed as an argument to the APPSPACK::Solver and may be customized by
the user. APPSPACK 4.0 contains two concrete implementations of the executor, APPSPACK::
Executor::Serial and APPSPACK::Executor::MPI.
The serial executor does exactly as its name implies— executes function evaluations one at a
time. In other words, the size of the pending queue P is exactly one and all evaluations are performed
immediately on the current processor (because there are no workers). The spawn operation calls
13
Page 14
the evaluator (described in Section 2.8) directly. An example of an APPSPACK executable for
serial mode is shown in Figure 4. While there may be some situations where the serial version of
APPSPACK is useful, this mode is provided primarily for testing purposes.
// Construct empty parameter list
APPSPACK::Parameter::List params;
// Parse input file to fill parameter list
// (argv[1] contains the name of the input file)
APPSPACK::parseTextInputFile(argv[1], params);
// Construct evaluator object using parameters from the input file
APPSPACK::Evaluator::SystemCall evaluator(params.sublist("Evaluator"));
// Construct serial executor using the justconstructed evaluator
APPSPACK::Executor::Serial executor(evaluator);
// Construct bounds object using parameters from the input file
APPSPACK::Constraints::Bounds bounds(params.sublist("Bounds"));
// Construct solver object using parameters from the input file and
// the just created executor and bounds.
APPSPACK::Solver solver(params.sublist("Solver"), executor, bounds);
// Solver the optimization problem
solver.solve();
// Access the answer
vector<double> x = solver.getBestX();
bool isF = solver.isBestF();
double f = solver.getBestF();
Figure 4. Calling APPSPACK in Serial Mode
In contrast, the MPI executor spawns function evaluations to workers that execute as separate
processes. In this version, the worker processes must be started independently in the main program.
Figure 5 contains a documented example of a default executable for the MPI manager, and Figure 6
gives a corresponding MPI worker executable. Both of these executables can be customized by the
user. The parallel version of APPSPACK is preferred and is how it is intended to be used.
2.8Evaluators
The actual objective function evaluation of the trial points is handled by the abstract interface
defined in the APPSPACK::Evaluator::Interface class. This structure allows the user to either
use the provided default evaluator or to create a customized one. The default evaluator can be
found in the APPSPACK::Evaluator::SystemCall class, and it works as follows: A function input
file containing the point to be evaluated is created. Then, an external system call is made to the user
provided executable that calculates the function value. After the function value has been computed,
the evaluator reads the result from the function output file. Finally, both the function input and
output files are deleted. This process is illustrated in Figure 7. Information regarding the user
14
Page 15
// Construct empty parameter list
APPSPACK::Parameter::List params;
// Parse input file to fill parameter list
APPSPACK::parseTextInputFile(argv[1], params);
// Send the evaluator information to each worker
APPSPACK::GCI::initSend();
params.sublist("Evaluator").pack();
for (int i = 0; i < nWorkers; i ++)
APPSPACK::GCI::send(APPSPACK::Executor::MPI::Init, i+1);
// Construct MPI executor.
// The evaluator will be constructed on the workers and is not passed
// as an argument to the executor.
APPSPACK::Executor::MPI executor;
// Construct bounds object using parameters from the input file
APPSPACK::Constraints::Bounds bounds(params.sublist("Bounds"));
// Construct solver object using parameters from the input file and
// the just created executor and bounds.
APPSPACK::Solver solver(params.sublist("Solver"), executor, bounds);
// Solver the optimization problem
solver.solve();
// Send a termination command to each worker
APPSPACK::GCI::initSend();
for (int i = 0; i < nWorkers; i ++)
APPSPACK::GCI::send(APPSPACK::Executor::MPI::Terminate, i+1);
// Access the answer
vector<double> x = solver.getBestX();
bool isF = solver.isBestF();
double f = solver.getBestF();
Figure 5. Calling APPSPACK in MPI Mode (Manager)
15
View other sources
Hide other sources
 Available from Genetha A Gray · May 27, 2014
 Available from Genetha A Gray · May 31, 2014
 Available from psu.edu