Model Driven Provisioning: Bridging the Gap Between Declarative Object Models and Procedural Provisioning Tools.
ABSTRACT Today’s enterprise data centers support thousands of mission-critical business applications composed of multiple distributed
heterogeneous components. Application components exhibit complex dependencies on the configuration of multiple data center
network, middleware, and related application resources. Applications are also associated with extended life-cycles, migrating
from development to testing, staging and production environments, with frequent roll-backs. Maintaining end-to-end data center
operational integrity and quality requires careful planning of (1) application deployment design, (2) resource selection,
(3) provisioning operation selection, parameterization and ordering, and (4) provisioning operation execution. Current data
center management products are focused on workflow-based automation of the deployment processes. Workflows are of limited
value because they hard-code many aspects of the process, and are thus sensitive to topology changes. An emerging and promising
class of model-based tools is providing new methods for designing detailed deployment topologies based on a set of requirements
and constraints. In this paper we describe an approach to bridging the gap between generated “desired state” models and the
elemental procedural provisioning operations supported by data center resources. In our approach, we represent the current
and desired state of the data center using object models. We use AI planning to automatically generate workflows that bring
the data center from its current state to the desired state. We discuss our optimizations to Partial Order Planning algorithms
for the provisioning domain. We validated our approach by developing and integrating a prototype with a state of the art provisioning
product. We also present initial results of a performance study.
- SourceAvailable from: guanotronic.com
Article: Statement of Research Interests[Show abstract] [Hide abstract]
ABSTRACT: et of test examples, or by citing some statistics for these particular outputs. Such an ad-hoc approach does not characterize the performance under various input instances. I am working on the development of techniques for performance evaluation. In case of PS, I localized the sources of systematic and random errors in the results and quantified the performance in terms of confidence intervals in each of the coordinates of the reconstructed points . 2. Teleimmersion (TI). Developments in technology and advances in research in the ares of computer graphics, machine perception, and high performance networks enable the creation of a TI in shared virtual reality environments. As an experimental scientists, I am interested in bringing the real world (i.e. 3-D models based on real dynamic scenes) into a virtual reality environment, thus merging virtual objects and 3-D models of a remote real world . The mo04/1999;
- [Show abstract] [Hide abstract]
ABSTRACT: To deliver 3-tier applications as a Service in the Cloud state-related constraints crossing Infrastructure- and Software as a Service boundaries need to be managed. By automating the lifecycle of applications like databases, load balancers, and web application servers rich SaaS business services can be provided in the Cloud. We propose an object oriented planning approach based on state constraints to plan for changes of SaaS and IaaS components in the Cloud. We evaluate techniques for fast storing and restoring of large object oriented Configuration Management Databases and show that enforcing constraints in a procedural instead of a declarative way offers huge performance improvements. The advantages of our approach lie within the tight integration of the planning algorithm with object oriented models frequently used for Configuration Management Databases. In addition to that, the algorithm scales to a large number of nodes and preserves its runtime even for large, heavily loaded data centers.IEEE International Conference on Cloud Computing, CLOUD 2010, Miami, FL, USA, 5-10 July, 2010; 01/2010
Conference Paper: Combining horizontal and vertical composition of services.[Show abstract] [Hide abstract]
ABSTRACT: Service composition is a well-established field of research in the service community. Services are commonly regarded as black boxes with well-defined interfaces that can be recursively aggregated into new services. The black-box nature of services does not only include the service implementation but also the middleware and hardware to run the services. Thus, service composition techniques are typically limited to choosing between a set of available services. In this paper we keep the black-box nature and the principle of information hiding for the service implementation, but break up services vertically. By introducing vertical service composition, we allow services to be provisioned on the right middleware when they are requested, thus making service-binding more powerful as services with the desired quality of service can be provisioned on demand. We introduce the concept of vertical service composition and present an extension to an enterprise service bus that implements the concept of vertical service composition by combining concepts from provisioning with those of (dynamic) service binding.IEEE International Conference on Service-Oriented Computing and Applications, SOCA 2010, 13-15 December 2010, Perth, Australia; 01/2010
Model Driven Provisioning: Bridging the Gap
Between Declarative Object Models and
Procedural Provisioning Tools
Kaoutar El Maghraoui1, Alok Meghranjani2, Tamar Eilam3,
Michael Kalantar3, and Alexander V. Konstantinou3
1Dept. of Computer Science, Rensselaer Polytechnic Institute, Troy, NY, USA
2´Ecole Polytechnique F´ ed´ erale de Lausanne, Lausanne, Switzerland
3IBM T.J. Watson Research Center, Hawthorne, NY, USA
Abstract. Today’s enterprise data centers support thousands of
mission-critical business applications composed of multiple distributed
heterogeneous components. Application components exhibit complex de-
pendencies on the configuration of multiple data center network, middle-
ware, and related application resources. Applications are also associated
with extended life-cycles, migrating from development to testing, stag-
ing and production environments, with frequent roll-backs. Maintaining
end-to-end data center operational integrity and quality requires care-
ful planning of (1) application deployment design, (2) resource selection,
(3) provisioning operation selection, parameterization and ordering, and
(4) provisioning operation execution. Current data center management
products are focused on workflow-based automation of the deployment
processes. Workflows are of limited value because they hard-code many
aspects of the process, and are thus sensitive to topology changes. An
emerging and promising class of model-based tools is providing new
methods for designing detailed deployment topologies based on a set
of requirements and constraints. In this paper we describe an approach
to bridging the gap between generated “desired state” models and the
elemental procedural provisioning operations supported by data center
resources. In our approach, we represent the current and desired state
of the data center using object models. We use AI planning to auto-
matically generate workflows that bring the data center from its current
state to the desired state. We discuss our optimizations to Partial Or-
der Planning algorithms for the provisioning domain. We validated our
approach by developing and integrating a prototype with a state of the
art provisioning product. We also present initial results of a performance
Today’s enterprises are increasingly reliant on network-based services to imple-
ment mission-critical business processes. A typical enterprise supports thousands
of business applications, composed of numerous heterogeneous distributed com-
ponents, and deployed in multiple large data centers. The collection of business
M. van Steen and M. Henning (Eds.): Middleware 2006, LNCS 4290, pp. 404–423, 2006.
c ? IFIP International Federation for Information Processing 2006
Model Driven Provisioning 405
applications is in a constant state of flux. New applications are developed, tested
on different test environments, staged and rolled into production. Existing appli-
cations are continuously updated, and often rolled-back. Data center operations
personnel must provision and configure multiple networked environments and
deploy applications into them. The capabilities of these environments have also
evolved from basic services such as routing, naming, and hosting, to higher-level
middleware services such as directory, messaging, and storage. Maintaining data
center operational integrity and quality has thus become an extremely chal-
lenging task. In the absence of end-to-end operational methodologies and tools,
enterprises and their customers are exposed to significant operational cost and
While operators enjoy a large set of tools to perform local configuration tasks,
they face major challenges in deploying complete applications. Current method-
ologies and tools provide fragmented and incomplete support for the end-to-end
application deployment process. This process can be broken into four major
logical steps, representing different domains of expertise. First, a deployment so-
lution must be designed that satisfies functional application deployment require-
ments and non-functional deployment goals. Second, resources must be selected
that can be used to implement the solution. Resource selection must take data
center capabilities and constraints into account. Third, an ordering of bound pro-
visioning operations must be established to bring the data center from its current
to the desired state. Complex constraints exist between configuration parameters
across the various tools and in provisioning operations that are far apart in the
ordering sequence . Implied or poorly documented ordering interdependencies
are typically discovered in the process of deploying an application. Forth, the
selected operations must be invoked across different management platforms and
domains. Operation execution status must be monitored, and operational errors
A new class of cross-platform management products, such as IBM Tivoli Pro-
visioning Manager (TPM), has emerged to address the bottom-up challenges
of operating heterogeneous data centers. These provisioning technologies offer
a large set of automation packages that expose a uniform data access and up-
date layer to heterogeneous management platforms. In addition, they provide
a platform for programming workflows for higher-level provisioning operations.
Users can create workflows invoking primitive provisioning operations or other
workflows to automate the deployment of a business application. Typically, such
workflows statically encode significant aspects of the application design, resource
selection, and operational ordering choices tied to a particular deployment envi-
ronment. Development, testing and maintenance of such workflows in a chang-
ing environment is a significant challenge. Since these workflows are specific to
a deployed application and a target environment, the potential level of reuse is
minimal. Thus, the amortized complexity is not reduced.
More recently, a new class of model-based tools has emerged to address the
top-down challenges of designing and binding applications to data center re-
sources. In these tools, the current data center state and the desired deployment
406K. El Maghraoui et al.
solution are both described declaratively using object-relationship models. These
tools aid in the construction and validation of a model describing the desired
solution, which may vary in its degree of concreteness. In particular, resource
instances may or may not be identified by it. In [3,4] we described the design
and implementation of such a tool, and the methods we used to guarantee that
the deployment topology satisfies an input set of requirements and constraints.
While these tools address the design and resource selection challenges, their out-
put is a declarative model, and the generalized task of realizing this model using
existing provisioning tools is an open problem.
We believe that representing configuration knowledge in object models will
offer significant advantages and will be the basis for the next generation of con-
figuration management tools . Models provide easier visualization, concep-
tualization, extensibility, componentization, standardization, and reuse. Most
technologies in this space are moving towards model-based solutions. However,
there exists an inherent mismatch between a declarative model that is the basis
for these modeling tools and the provisioning technologies that are in essence
procedural. Moreover, the granularity does not match: models are typically fine
grained, while provisioning operations are coarser grained. In particular, provi-
sioning operations may have a complex effect on multiple resources. In model-
ing terms, multiple objects and relationships may change in a model describing
the state of the data center before and after an execution of a provisioning
In this paper we describe an approach which bridges the gap between the
declarative model of a solution and the procedural provisioning operation tool-
ing needed for its implementation. We use models to declaratively describe the
required solution, as well as the operational capabilities of existing provision-
ing platforms. We then employ a planning algorithm to automatically infer the
partial order of provisioning operations and their inputs to deploy a given ap-
plication in a data center. The generated workflows maintain operational con-
straints while verifiably provisioning the desired data center state. Our approach
supports the seamless integration of existing automation tools specializing in ap-
plication solution design, resource selection, and cross-platform provisioning. We
based and evaluated our models and algorithms on the capabilities of a state of
the art provisioning product, in customer use.
The structure of the paper is as follows. In Section 2, we present an overview
of our approach and architecture for model driven deployment planning. Next,
in Section 3, we present a formal model for applying planning to the problem
domain. In Section 4, we describe the planning algorithm and how we optimized
it for our particular usage of deployment planning. In Section 5, we describe our
prototypical implementation and integration with TPM , a state of the art
provisioning product. In Section 6, we present empirical results of a multi-tier
network provisioning experiment. Last, in Section 7, we review the related work,
and in Section 8, we summarize the work and discuss future challenges.
Model Driven Provisioning407
2.1Background: Data Center Operations Today
Data center operators use a large number of automation and configuration tools
to deploy distributed applications and services on a set of managed resources.
Every such tool provides a set of automated resource management functions,
that we term provisioning operations. To accomplish a particular configuration
task, the operator must identify a set of provisioning operations provided by
the collection of available tools, instantiate them correctly, and execute them in
an appropriate order. Hundreds of low-level provisioning operations may be re-
quired to deploy a single application. For example, servers must be selected from
a free pool, network switches, firewalls and load balancers must be configured,
operating systems, middleware, and application components must be installed
and/or configured, and monitoring must be enabled. Some of these tasks, such
as the selection of resources, are performed manually, others with tooling assis-
tance. The ordering and parameterization of provisioning operation invocations
is determined by operators in an ad hoc manner. Operators typically rely on
past experience, product manuals, existing scripts and other unstructured and
informal data sources.
Individual provisioning operations may incorporate complex logic. A provi-
sioning operation often makes assumptions about the state of the affected re-
sources and about other resources connected to them. Upon invocation, it may
perform a large number of fine grained configuration actions effecting the state
of a number of resources in the data center. For example, a provisioning oper-
ation to install an operating system may need to configure a DHCP server and
a network image server, in addition to the target system of the installation. In
secure environments, the OS install operation might have to configure a number
of network devices to ensure connectivity between the install server and the tar-
get server. To determine a successful order of executions, operators must fully
understand the preconditions and effects of each of the provisioning operations
and their interdependencies. Due to the aforementioned complexities, operators
often rely on step-by-step trial and error operation. Even a simple application
migration from the developer’s workstation to a testing environment can become
a challenge, with studies indicating it accounts for 35% of the testing time1.
2.2A Case for Model Driven Deployment Using Planning
Increasingly, object models are used in order to formally describe resource con-
figuration state. The objects in these models are typically typed and associated
with attributes. For example, the Management Information Base (MIB) of an
IP system will contain an object for each IP network interface, with attributes
such as IP address and netmask. The IP interface node will also have a relation-
ship to the network interface card (NIC) object on which it is defined. These
configuration models can be navigated and queried at a fine level of granularity.
1Theresa Lanowitz, speaking at a Mercury Users Conference, 2004.
408K. El Maghraoui et al.
While configuration models offer uniform access and navigation, their update
functions are typically not uniform or consistent. A single operation may have
multiple parameters, pre-conditions and post-conditions. For example, the op-
eration to configure an IP interface may take multiple parameters such as the
IP address, netmask, NIC, and default gateway, where the IP address must be
unique and must match the netmask, the NIC must be enabled and connected
to a link-layer network associated with the netmask, and so on. The execution
of the operation may result in changes at multiple attributes and nodes in the
The complexity of configuration operation parameterization and dependency
ordering, necessitates the use of advanced algorithms for inferring and generat-
ing correct sequences of provisioning actions, termed workflows. The workflow
generation problem can be naturally reduced to the AI planning problem. A
planning system synthesizes a course of actions to change the world from its
current state to a desired goal state. A planning domain defines a set of atomic
actions that are capable of changing the state of the world. Each action can only
be executed under some particular conditions of the world termed preconditions.
Each action has certain effects on the state of the world. A planner generates a
plan: a sequence of actions that will bring the world from its initial state to the
Use of planning for workflowgeneration allows users to focus on the declarative
expression of data center resources, desired state and operation models. Models
can be defined by different users, supporting separation of concerns across re-
source types and operational domains. For example, a CISCO router expert may
model the pre-conditions and post-conditions of a CISCO IOS router configu-
ration operation. A deployment expert may add a constraint that a route must
exist between the boot server and a system being rebooted. These models can be
created once, and reused many times to automatically generate multiple work-
flows to deploy multiple applications in different networked environments. Thus,
the amortized complexity of managing the enterprise data center is significantly
reduced using this approach.
Our approach to deployment planning and execution can be summarized as
follows. (1) we formally capture both the current state of the data center, and the
desired deployment solution using object-relationship models, (2) we formally
capture the pre-conditions and effects of a key set of provisioning operations
provided by the available tools using propositional logic, (3) we employ partial
order planning algorithms to automatically generate sequences of provisioning
operations to bring the data center from its current state to the desired state. We
optimize partial order planning to the area of provisioning by utilizing domain
knowledge and data center characteristics.
The detailed architecture of our approach is depicted in Fig. 1. The current
state of data center resources is maintained in a configuration database in the
form of an object-relationship model. Automation tools are integrated into the
system by a specialized adapter. This adapter provides an abstraction layer and
a common access layer to execute the functions that are provided by the tool.
Model Driven Provisioning409
Orchestration Workflow Engine
Workflow Generator (Planner)
Data Center Managed Resources
(desired data center state)
In addition, the pre-conditions and effects of key provisioning operations are
formally modeled as predicates and transformers over the data center state and
kept in a repository.
A workflow orchestration framework provides the means to author workflows,
encoding sequences of invocations of provisioning operations, and to manage
their execution. These worflows, in addition to invoking operations on the tools,
update the configuration database, based on the expected or actual result of
the execution. This is necessary since the tools are agnostic of the configuration
database. Note that this will always be an approximation of the actual state of
the data center. Discovery techniques can be employed to fix any inaccuracies
in the state of the data center as it is recorded in the configuration database.
A workflow generator (planner) receives three inputs: the current state of
the data center, the desired deployment topology, and the available provisioning
operations. The desired topology is generated using a modeling tool, such as .
The workflow generator employs a planning algorithm to automatically generate
orchestration workflows. The workflow generator component is the focus of this
paper. The rest of paper focuses on the design, implementation, algorithms, and
empirical studies of this architectural component.
3Generating Workflows Using Planning
Given a model describing the desired deployment state of an application, the
task of workflow generation involves identifying the provisioning operations to
be performed, binding of operational parameters, and analyzing ordering de-
pendencies between operations. The set of available provisioning operations is
determined by the provisioning technology. In this section, we describe how we
can use planning methods to generate the workflows by mapping the models rep-
resenting current and desired state to first order logic that is the input to most
planners, and modeling the provisioning operations as planner actions. Sect. 4
410K. El Maghraoui et al.
will focus on the planning algorithm and our adjustments and optimizations
necessary for it to work well in this domain.
3.1Problem Domain Modeling
State Modeling. Any resource object model (in fact any network model )
can be simply mapped to first-order logic. For a given object model, consisting of
typed nodes and relationships containing attributes, the following construction
will generate an equivalent planning initial or goal state:
– For each object instance N of type T, add the following predicates:
• (exists N), to express that the object is in the created state.
• (T N), to express the type of the object.
• For object models with support for inheritance, add (T1 N), (T2 N),
... predicates for each supertype Tiof type T.
– For each relationship instance E of type T, between N1 and N2, add the
• (established E N1 N2), to express that the relationship is in the es-
• (T E), to express the type of the relationship.
• For object models with support for relationship type inheritance, add (T1
E), (T2 E), ... predicates for each supertype Tiof relationship type T.
– For each object or relationship O, and an attribute A declared in type T
with value V , add the following predicates:
• (set O A), to express that the attribute is set.
• (T.A O V ), to express the attribute’s value.
The above rules are used to translate both the initial data center model and
that of a desired topology (which represents the desired state of all the resources
in the data center) into a first-order representation.
Figure 2 shows a sample object-relationship configuration model of a data
center server and its logic representation. The server contains a network interface
set sp001 spNumber
SwitchPort.spNumber sp001 14
established ncts01 sp001 en0
Fig.2. Example object model and logical representation
Model Driven Provisioning411
: parameters (?switch - Switch
?sp – SwitchPort
?vlan1 - Vlan
?vlan2 – Vlan
?vcs1 - VlanContainsSp
?vcs2 - VlanContainsSp)
: precondition (and (exists ?switch) (exists ?sp) (exists ?vlan2)
(established ?scs ?switch ?sp)
(established ?vdos ?vlan2 ?switch)
(established ?vcs1 ?vlan1 ?sp)
(set SwitchPort.spNumber ?sp) (set SwitchPort.spModule ?sp)
(set Vlan.vlanNumber ?vlan2))
:effect (and (established ?vcs2 ?vlan2 ?sp)
(not (established ?vcs1 ?vlan1 ?sp))
Fig.3. PDDL specification for the moveSwitchPortToVlan operation
card (NIC) which is connected to a switch port on switch sw01. The switch port
is also configured to be a member of the virtual LAN (VLAN) vlan1 defined on
sw01. The SwitchPort type is inherited in our type system from the Nic type.
vlan1 has an attribute vlanNumber with value 201.
Action Modeling. Provisioning operations are modeled as planner actions.
Typically, provisioning actions are implemented imperatively, thereby requir-
ing additional declarative modeling of pre-conditions and post-conditions. The
Planning Domain Definition Language (PDDL)  is a common language for
expressing planning domains. Preconditions for each action may express restric-
tions on the life-cycle state of entities (e.g. exists), graph structure, and attribute
values. The effects of the actions are similarly expressed.
An example of a configuration operation model expressed in PDDL is shown
in Fig. 3. The operation moveSwitchPortToVlan configures a switch to assign a
switch port into a particular VLAN. This allows the computer system connected
to the port to communicate with other computer systems in that VLAN as if they
were on the same local network. In order to move a switch port into a VLAN,
the switch port, the switch and the VLAN must all exist and be interconnected.
The preconditions clause in Fig. 3 express these requirements. The effects clause
indicates that after the execution of this action, the switch port will be contained
in the VLAN. For simplicity, we omitted the type predicates for both objects
and relationships. They are implied from the definition of the parameters.
Planning algorithms are a class of search algorithms. They basically search and
backtrack various possible plans until they find a solution. Classic planners adopt
one of two approaches: searching the world state space or searching the plan
space. In the first approach, the search space consists of a graph whose nodes
412K. El Maghraoui et al.
represent the state of the world and whose edges represent the execution of ac-
tions (e.g. GraphPlan ). The planner can search world space starting from the
initial state (progression planners) or starting from the goal state (regression
planners). In the second approach, each node in the graph represents a partial
plan, and each edge represents a plan refinement operation. Of these, some al-
gorithms generate totally ordered plans (Total Order Planning), while others
generate partially ordered plans (Partial Order Planning or Least Commitment
For the task of provisioning workflow generation, we selected partial order
planning (POP) for the following reasons:
1. A partially ordered plan can be efficiently executed in parallel. Provisioning a
distributed application typically requires configuring multiple systems. These
configurations can usually be done in parallel.
2. In any given data center state, there may be many possible actions that can
be executed. The number of such actions is proportional to the size of the
data center. Consequently, world state search approaches will have a high
branching factor. A high branching factor is likely to increase planning cost.
3. Partially ordered planning algorithms are efficient when the number of pos-
sible actions to fulfill a given condition is small. This is the common case
for distributed application provisioning: typically only a few provisioning
operations will produce a particular configuration of a given resource.
3.3Partial Order Planning
A partial order planner searches the plan space. Each node in the search space
represents a partial plan while edges represent plan refinement operations. We
briefly review partial order planning; for a more detailed description see, for
A partial plan in POP is a set of action steps S0,S1,...,Sf and a set of
ordering constraints Si< Sjwhich indicate the causal order of the action steps.
In addition, the following meta information is maintained: (1) a set of causal
links Si →cSj that record that precondition c of step Sj is achieved by step
Si, (2) a set of open conditions which consist of action preconditions that still
remained to be achieved, and (3) a set of unsafe links Si →cSjindicating that
precondition c is deleted by some step Skin the partial plan.
Partial order planning begins with an initial unfinished plan comprising two
dummy steps: Start Step S0and Finish step Sf. S0is a step with no preconditions
and whose effects represent the world in the initial state. Sf is a step with no
effects and whose preconditions represent the world in the goal state. This initial
plan is iteratively refined by applying plan refinement operations also termed flaw
resolutions. Flaw resolutions fall into two categories:
unachieved precondition of an action already added to the partial plan. An
open condition can be achieved by adding a new action to the partial plan
or by reusing an action already in the plan.
ConditionAchievement: An open condition represents a
Model Driven Provisioning413
2. Unsafe Links Resolution: An unsafe link indicates that an achieved con-
dition may be invalidated by another action. In this case, ordering the actions
avoids the conflict.
A partial plan is a complete plan when there are no flaws in it. A partial
planner continues to refine the different partial plans generated until a complete
plan is found or all the different possibilities have been tried and no solution is
found. Observe that a POP planner generates a plan backwards by identifying
actions that achieve the preconditions of actions already in the plan.
General-purpose planning algorithms suffer from poor scalability. The use of
domain-specific knowledge is critical to developing practical planners . We
show in subsequent sections, how we have exploited the nature of the domain of
network configurations in distributed application provisioning to achieve signif-
icant improvements in the efficiency of POP for this purpose.
4Optimizing POP for Provisioning
4.1 Domain Characteristics
The domain of distributed application deployment poses efficiency challenges to
the generic partial order planning algorithm:
Complex Provisioning Operations. Provisioning operations tend to be com-
plex, ofter performing multiple configuration tasks. Consequently, even simple
operations frequently have several parameters and effects. Their preconditions
also tend to have many clauses. When adding an action to a partial plan, each
parameter that must be instantiated acts as a multiplier to the number of pos-
sible variable instantiations, resulting in a high branching factor.
Data Center Size. A typical data center manages hundreds, if not thousands,
of resources. This means that for a given action parameter, the planner needs to
consider a large number of possible values. For example, when trying to instan-
tiate a server variable in an action, there may be hundreds of possible instanti-
ations. In partial order planning, if we naively attempt to fully instantiate each
action that is added to a partial plan, the branching factor will be prohibitive.
Constrained Resource Modifications. Resources can be configured only in
a limited number of ways, and data center policy typically introduces even more
configuration constraints. For example, the set of NICs that a server contains
is typically fixed. In addition, once a server is wired into a data center, its
relationship (through its NIC) to a switch port on a particular switch is typically
fixed. Consequently, a resource typically has a number of fixed relationships
with other resources. Many provisioning operations tend to be local in nature:
they operate on groups of closely related parameters. In subsequent sections we
explore how a planner can take advantage of the fixed relationships between
resources to efficiently instantiate parameters.
414K. El Maghraoui et al.
Runtime Parameter Determination. Not all parameters for provisioning
operations are available at planning time. Some are only available at deploy-
ment time. For example, to enable communication, it is necessary to configure
a server’s network interface with an IP address. It is usually not possible to
select any unused IP address; the selection is constrained by a runtime data cen-
ter policy. As a consequence, at planning time, it is not possible to instantiate
all variables. We discuss how we deal with this, using a concept that we term
deferred instantiation, in Sect. 4.2 below.
4.2Partial Order Planning for Provisioning
Prioritized Flaw Selection. Partial order planning proceeds by iteratively
selecting flaws to resolve. This selection can be ordered to improve planning
efficiency. In our workflow generator, we, as in  and , resolve unsafe links
before open conditions. With regards to resolving open conditions, we use the
following priority (in a descending order):
1. Open conditions of the form (exists ?<type>).
2. Fully instantiated open conditions.
3. Other partially instantiated open conditions.
Our highest priority is to instantiate unknown resources. We do so for two
reasons: first, such open conditions have only one uninstantiated variable, helping
to reduce branching. Second, once resources are bound, it is more likely that open
conditions representing resource relationships and attribute values will be more
constrained, again reducing branching.
Fully instantiated open conditions are given preference compared to partially
instantiated open conditions because they constrain the problem more. They are
least likely to introduce the uninstantiated open conditions.
When comparing two open conditions which are either fully instantiated or
which are both partially instantiated, the flaw selection algorithm counts the
number of partial plans that will be created to resolve each open condition. The
open condition that generates the fewest new partial plans will be selected.
Condition Driven Variable Instantiation. Recall that one class of flaw res-
olutions are open condition achievement operations. These operations involve
adding a new action to a partial plan or reusing an existing action in a partial
plan to achieve the precondition of another action in the partial plan. When
adding actions we can choose to instantiate all of its parameters with specific
values or instantiate only some of them. If we choose to instantiate all of the pa-
rameters, we create a new partial plan for each combination of fully instantiated
variables. Recall that provisioning operations typically have a large number of
parameters and that data centers manage a large number of resources. Conse-
quently, a large number of new partial plans will be generated. In search terms,
the branching factor will be high. On the other hand, we may leave parameters
uninstantiated until a consistency threat necessitates instantiation. If we leave
parameters uninstantiated, the branching factor remains low, however, planning
Model Driven Provisioning415
becomes more complex, as it is necessary to maintain variable binding constraints
that specify whether two parameters are the same. Further, it is necessary to
implement a unifier that takes into account the variable binding constraints to
identify valid tuples of parameters. Unsafe link detection and action matching
become more complex as well.
We adopted a hybrid approach that reduces the branching factor but which
minimizes the complexity of the implementation and the performance overhead.
In our approach, when adding a new action we instantiate only the parame-
ters that are needed in order to satisfy the open condition. Note that adding
an action may result in new open conditions corresponding to the unsatisfied
preconditions of this new action with uninstantiated variables. Unlike the lazy
approach described above, we do not allow these uninstantiated variables to be
propagated to new actions at a subsequent step. Instead, when we select an open
condition that is uninstantiated, we generate instantiations at that time. Our ap-
proach reduces the branching factor because the number of variables in an open
condition is low for provisioning (typically one or two). Further, the need for
variable binding constraints is minimized because uninstantiated variables are
not propagated. This significantly reduces the complexity of the implementation
and the performance overhead.
Model Guided Variable Instantiation. We take advantage of knowledge on
resource and data center configurability constraints to minimize the number of
tuples created when binding variables in an open condition. For example, the
relationships between servers and their NICs, and also typically the relationships
between the NICs and the switch ports to which they are connected, are all fixed.
These fixed relationships limit the number of resources that need to be considered
when instantiating an action.
As an example, consider the provisioning operation addNetworkInterface
shown on the left side of Fig. 4, and its precondition (established ?scn ...),
where scn is a relationship of type SystemContainsNic. Without our strategy,
the planning algorithm would generate all pairs of systems and NICs in the
data center, 8 combinations will be generated for the data center model piece
shown in Fig. 4 (types and identities of relationships are omitted for simplicity).
Most would lead to a search dead end since no action exists that can change
the fixed relationships between servers and their NICs. Not only are a large
number of possible instantiations generated, but they are not immediately elim-
inated from consideration. Using our strategy, on the other hand, only pairs
connected by a relationship will be considered. Specifically, only the following
pairs will be generated by the planning algorithm when instantiating this ac-
tion: (EJB-server, nic4), (EJB-server, nic5), (Data-server ,nic1), and
(Data-server, nic2). Clearly, if one of the variables in this example were al-
ready instantiated, the number of tuples is further reduced. As a further en-
hancement of this strategy, we not only look at the preconditions of the current
action when instantiating variables, but also at other preconditions, associated
with relationships that are know to be fixed, where the variables appear.
416K. El Maghraoui et al.
Fig.4. Specification of the addNetweorkInterface provisioning operation and part of
initial state showing two servers, their NICs and several IP addresses
Deferred Variable Instantiation. A deployment topology describes how re-
sources in a data center should be configured. While the deployment topology
identifies all the resources needed, it may not have fully selected them. Recall
that a data center may implement policies that prevent some types of resources
from selection until deployment time. For example, to configure a network in-
terface, it is necessary to have an IP address. While the planner knows, based
on the data center model, what IP addresses are in use, it does not have control
over the selection of IP addresses, as data center policies typically determine
the selection at deployment time using a provisioning operation. This restric-
tion prevents the planning system from instantiating the IP address in other
network configuration actions, such as to configure network interfaces, routing,
and access control. To address this challenge, we introduce the concept of de-
ferred variable instantiation. For variables that can only be instantiated at de-
ployment time, we create placeholders that represents the instantiated variables.
Such a substitution can, however, take place only when the following conditions
1. There is a provisioning operation (action) that can create a new unique
instance of the required variable. For example, to resolve an IP address, the
provisioning system may have an operation getIPAddress that generates a
valid IP address.
2. The variable does not change in value once it is created.
The placeholder represents the output of a particular provisioning operation.
It is treated as a read only instantiated variable that can be used as a parameter
to other operations.
Model Driven Provisioning417
5 Prototype Implementation
We implemented a prototype of the workflow generator architecture and plan-
ning algorithm described in the previous sections. For the role of the workflow
generator, we developed a custom Java-based partial order planner. The plan-
ner was implemented with configurable support for our domain specific variable
instantiations, flaw selection, and deferred variable instantiation. For the role of
the workflow engine, we used the IBM Tivoli Provisioning Manager (TPM) 
v.3.1 product. TPM workflows are parameterized with strongly typed objects
defined in a data center model (DCM). The DCM schema is DMTF CIM-
based . DCM instance data is populated manually, or automatically by dis-
covery tools. TPM provides a device abstraction layer whereby logical device
operations (LDOs) are declared against DCM types. This layer enables users
to define workflows that can operate over different vendor implementations of
a logical device. DCM objects are bound to device drivers that bundle device-
specific implementations of logical device operations. We modeled a subset of
the TPM LDOs relating to network configuration, as planner actions. We also
implemented an importer from the DCM object model to logical representation.
Finally, we implemented an Eclipse-based graphical user interface, with views
for planner operations, initial and goal states, and generated operation partial
Our prototype was also integrated with the SPiCE (Service Plan Composi-
tion Engine) model-driven data center design tool . Using SPiCE, users could
customize a logical application structure with deploy-time choices, and automat-
ically generate the desired state of the data center. Our worflow generator would
then be invoked, in the same Eclipse shell-sharing environment, to generate the
the partial order of TPM LDOs required to provision the data center changes.
A TPM workflow exporter was implemented to convert the planner’s output
to the TPM workflow language. Generated TPM workflows were submitted for
execution to the TPM deployment engine.
We evaluated our prototype using the desired network structure of a three-
tier clustered application consisting of a web, business logic, and data tier. An
example of this structure is depicted in Fig. 5. An external browser system was
defined to model remote access to the web tier. Browser traffic would be routed
through a firewall, connected to a load-balancer, spreading requests across the
servers in the web cluster. Web tier servers would invoke business logic functions
by routing traffic over an internal firewall. Load balancing on the business tier
would be performed at the application-level. Business tier servers were modeled
as dual-homed, connecting to the data tier through another firewall. The figure is
a screen shot of the SPiCE visualizer and depicts a filtered view over the desired
state topology. Server, router and load-balancer network interface card (NIC),
IP interface configuration, and routes were hidden. The switch and switch ports
over which the VLANs were defined were also hidden.
418K. El Maghraoui et al.
Fig.5. Sample three-tier application network structure
The desired state models were varied in the number of servers per cluster.
The models were generated by defining a SPiCE logical application structure
and varying the cluster size. For our infrastructure modeling, we created a pa-
rameterized DCM generator that created data centers with the requisite number
of switches, routers, load-balancers and servers. The number of NICs on each
device was also parameterized. The generated data center resources were instru-
mented by a simulator device driver provided by TPM.
We modeled the network device logical device operations (LDOs) for creating
a VLAN on a switch, assigning a port to a VLAN, creating IP network inter-
faces on systems (routers and servers), creating routes, access control lists, and
virtual IP addresses, creating clusters and free-pools, and adding servers to clus-
ters and free pools. We also modeled resource-selection operations for selecting
IP addresses/subnets, assigning tiers, and determining cluster expansion sizes
(for support of TPM dynamic orchestration features). Figure 6 lists a partial
workflow generated for a minimal topology. The workflow represents a serial ex-
ecution of the partial order generated by the planner (topological sort). It starts
by creating the spare pool, customer and subnet logical resources defined in the
desired state. Next it obtains a unique VLAN ID, and creates a new VLAN
with the specified ID in the switch identified in the desired state. It configures
the switch port to the newly created VLAN ID (must precede VLAN creation
on switch). Note that this operation required the switch module containing the
port as a parameter. This information was missing from the desired state. The
planner used the module port lookup LDO to obtain the required parameter.
We focus on scalability studies that show our domain specific enhancements,
described in Sect. 4.2, scale well and are better performing than the generic POP
algorithm. We present two scalability experiments using the workflow generator.
They investigated the scalability of the generator in terms of the infrastructure
size and the size of the three-tier application desired state.
First, we varied the number of available resources in the data center keep-
ing the number of resources in the desired state constant. Because the desired
topology was unchanged, the number of provisioning operations was constant,