An Adaptive Middleware for Supporting Time-Critical Event Response

Abstract

There are many applications where a timely response to an important event is needed. Often such response can require significant computation and possibly communication, and it can be very challenging to complete it within the time-frame the response is needed. At the same time, there could be application-specific flexibility in the computation that may be desired. This paper presents the design, implementation, and evaluation of a middleware that can support such applications. Each of the services in our target applications could have one or more service parameters, which can be modified, within the pre-specified ranges, by the middleware. The middleware enables the time-critical event handling to achieve the maximum benefit, as per the user-defined benefit function, while satisfying the time constraint. Our middleware is also based on the existing Grid infrastructure and Service-Oriented Architecture (SOA) concepts. We have evaluated our middleware and its support for adaptation using a volume rendering application and a Great Lake forecasting application. The evaluation shows that our adaptation is effective, and has a very low overhead.

Full text

Page 1

Cluster Comput (2009) 12: 87–100
DOI 10.1007/s10586-008-0071-x
An adaptive middleware for supporting time-critical event
response
Qian Zhu ·Gagan Agrawal
Received: 7 October 2008 / Accepted: 9 October 2008 / Published online: 6 November 2008
© Springer Science+Business Media, LLC 2008
Abstract There are many applications where a timely re-
sponse to an important event is needed. Often such response
can require significant computation and possibly communi-
cation, and it can be very challenging to complete it within
the time-frame the response is needed. At the same time,
there could be application-specific flexibility in the compu-
tation that may be desired.
This paper presents the design,implementation,and eval-
uation of a middleware that can support such applications.
Eachoftheservicesinourtargetapplicationscouldhaveone
or more service parameters, which can be modified, within
the pre-specified ranges, by the middleware. The middle-
ware enables the time-critical event handling to achieve
the maximum benefit, as per the user-defined benefit func-
tion, while satisfying the time constraint. Our middleware is
also based on the existing Grid infrastructure and Service-
Oriented Architecture (SOA) concepts. We have evaluated
our middleware and its support for adaptation using a vol-
ume rendering application and a Great Lake forecasting ap-
plication. The evaluation shows that our adaptation is effec-
tive, and has a very low overhead.
Keywords Self-adaptation · Time-critical event · Grid
middleware
Q. Zhu (?) · G. Agrawal
Department of Computer Science and Engineering,
Ohio State University, Columbus, OH 43210, USA
e-mail: zhuq@cse.ohio-state.edu
G. Agrawal
e-mail: agrawal@cse.ohio-state.edu
1 Introduction
There are many applications where a timely response to an
important event is needed. Often such response can require
significant computation and possibly communication, and
it can be very challenging to complete it within the time-
frame the response is needed. The resources available for
the processing may only be detected when the event occurs,
and may not be known in advance. At the same time, there
could be application-specific flexibility in the computation
that may be desired. For example, models can be run at dif-
ferent spatial and temporal granularities, or running all mod-
els may not be equally important. There could be a user pro-
videdbenefitfunction,whichcaptureswhatismostdesirable
to compute.
Inordertocompletecomputationwithinthepre-specified
time frame, while attempting to maximize the pre-specified
benefit function, numerous performance-related parameters
must be continuously tuned. As applications are complex
and dynamic in their behaviors and interactions, it is highly
desirable for them to be autonomic, i.e., self-managing and
self-optimizing, requiring only high-level guidance from ad-
ministrators [11]. This paper presents the design, implemen-
tation, and evaluation of such an autonomic middleware.
Our middleware supports applications that comprise a set
of services. Each of these services could have one or more
service parameters, which can be modified, within the pre-
specified ranges, by the middleware. Examples of such ser-
vice parameters could be the time step that decides the tem-
poral granularity of a model, or image resolution that de-
cides the computation’s spatial granularity.
The main functionality of our middleware is to enable the
time-criticaleventhandlingtoachievethemaximumbenefit,
as per the user-defined benefit function, while satisfying the

Page 2

88 Cluster Comput (2009) 12: 87–100
time constraint. We have given a formal model for the adap-
tationprocessinourframeworkbasedontheoptimalcontrol
theory [12]. Based on this formulation, we have developed
an autonomic adaptation algorithm. The algorithm includes
a learning phase, where the relationships between the values
of the service parameters and the computation time, bene-
fit function, and relative workload of each service are es-
timated. Furthermore, the algorithm detects global patterns
based on local adaptation to improve the efficiency.
The other goals in design of the middleware are related
to compatibility with grid and web services. We use the
existing Grid infrastructure and Service-Oriented Architec-
ture (SOA) concepts. Particularly, our system is built on
top of the Open Grid Services Architecture (OGSA) [6],
and uses its latest reference implementation, Globus 4.0.
The Web Services Resource Framework (WSRF) specifi-
cation supports efficient service management, with access
to stateful service resources. Furthermore, we enable easy
deployment and management of the application with min-
imum human intervention. This is done by supporting an
AutoServiceWrapper, which extracts information from the
application configuration file and wraps the code as an au-
tonomic service component. Service components cooperate
with each other during the processing.
Our middleware is also designed to use resources from
a heterogeneous distributed environment. The only require-
ments from a node to be used for executing one of the ser-
vices are (1) support for Java Virtual Machine (JVM), as the
middleware and wrapper are written in Java, and (2) avail-
ability of GT 4.0.
We have carefully evaluated our middleware and its sup-
port for adaptation using a volume rendering application and
a Great Lake forecasting application. The main observations
from our experiments were as follows:
• When handling a time-critical event, we were able to opti-
mize the benefit function within the pre-defined time con-
straint, with the parameters converging quickly to their
ideal values. Furthermore,our algorithm was significantly
more effective than a simple linear adaptation approach.
• The overhead of adaptation caused by our algorithm in
time-critical event handling is below 14%, when com-
pared to an ideal or optimal execution, which started with
parameter values that were dynamically chosen by our al-
gorithm.
• The overhead of the proposed algorithm in the learning
phaseisonly 5% forthevolumerenderingapplicationand
is around 10% for the Great Lake forecasting application.
The rest of the paper is organized as follows. We moti-
vate our work by two real applications in Sect. 2. The de-
sign of the adaptive middleware is described in Sect. 3. In
Sect. 4, we propose the system model and our autonomic
adaptation algorithm. Results from experimental evaluation
are reported in Sect. 5. We compare our work with related
research efforts in Sect. 6 and conclude in Sect. 7.
2 Motivating applications
This section describes two applications we are currently tar-
geting. Both applications require time-critical response to
certain events.
Volume rendering
jection of a large time-varying 3D data set (volume data) [5].
This volume data can be streaming in nature, e.g., it may be
generated by a long running simulation, or captured contin-
uously by an instrument. An example of the application is
rendering tissue volumes obtained from clinical instruments
in real-time to aid a surgery. Under normal circumstances,
the system invokes services for processing and outputs im-
ages to the user at a certain frame-rate. In cases where a no-
table event is detected in a particular portion of the image,
the user may want to obtain detailedinformation on that area
as soon as possible. For example, if an abnormality emerges
in a part of the rendered tissue image, the doctor will like to
do a detailed diagnosis in a timely fashion.
Time may be of essence, because of the need for altering
parameters of the simulation or the positioning of the instru-
ment. In obtaining the detailed information, there is flexibil-
ity with respect to parameters such as the error tolerance,
the image size and also the number of angles at which the
new projections are done.
Now, let us suppose that we can formally define a benefit
function, which needs to be maximized in the given amount
of time and with available resources. Let the set of all pos-
sible view directions be denoted as ?. Let Nb be the to-
tal number of data blocks in the dataset. For any given data
block i, the importance value [31] and the likelihood of be-
ing visited are denoted as I(i) and L(i), respectively. An-
other set of parameters includes the spatial error (SE) and
the temporal error (TE) [33]. Both of them should be close
to a pre-defined level, (SE0,TE0).
Then, the benefit function can be stated as:
involves interactively creating a 2D pro-
BenefitVR=
?
δ∈?
Nb
?
i=1
I(i)×L(i)
p
×e−(SE−SE0)(TE−TE0)
(1)
Intuitively, (1) implies that the user wants to view high-
quality images from all possible view directions. For each
view angle δ, the first factor impacting the quality of the fi-
nal image is captured by the sum of contribution of each
data block over the penalty of choosing non-beneficial
blocks (p). We further calculate the contribution of data

Page 3

Cluster Comput (2009) 12: 87–10089
block i from its importance value and the likelihood of be-
ing visited. The second part is related to the image quality,
involving the spatial and temporal errors, respectively. Al-
though none of the tunable parameters error tolerance or
image size is directly a variable in the benefit function, dif-
ferent choices of values from them would significantly im-
pact the benefit we can obtain.
Great Lake Forecasting System (GLFS)
rological conditions of the Lake Erie for nowcasting (for the
next hour) and forecasting (for the next day). Every second,
data comes into the system from the sensor planted along
the coastal line, or from the satellites supervising this partic-
ular coastal district. Normally, the Lake Erie is divided into
multiple coarse grids, each of which is assigned available
resources for model calculation and prediction. In a situa-
tion where particular areas in Lake Erie encounters severe
weather condition, such as a storm or continuous rain, the
experts may want to predict additional factors, by executing
other models. One possible goal may be managing sewage
disposal in this area in view of the severe weather.
There is a strict time constraint on when the solution to
these models are needed. However, while it is desirable to
run the models with high spatial and temporal granularity,
clearly there is some flexibility in this regard. For example,
such flexibility could be the resolution of grids assigned to
the model from a spatial view, or the internal and external
time steps deciding the temporal granularity of model pre-
diction. Furthermore, if computing resources available are
limited, running new models in certain areas may be more
monitors meteo-
critical than running those at other areas. Thus, we can for-
malize a benefit function as follows:
BenefitPOM=
?
w ×R +Nw×1
4R
?
×
M
?
i=1
P(i)
C(i)
(2)
w =
?1 if water level is predicted
0 otherwise
Equation (2) specifies that the water level has to be predicted
by the model within the time constraint since it is the most
important meteorological information. R is a constant value
for reward if this criterion is satisfied. It also gives credits to
other meteorological information outputs and the number of
outputs Nwhas to be maximized. Besides outputting useful
results, the user also wants the resources to be allocated to
the models with high priority. This is captured by getting the
ratio the model priority P(i) and its cost C(i).
3 Middleware design
This section describes the major design aspects of our adap-
tive and service-oriented middleware.
3.1 System architecture and design
The overall system architecture is illustrated in Fig. 1. An
application is implemented as a set of loosely-coupled auto-
nomic service components, with possible dependencies be-
tween them. Each service component is able to self-describe
Fig. 1 Overall design of the
adaptive service middleware

Page 4

90 Cluster Comput (2009) 12: 87–100
its interface and self-optimize its contribution to the over-
all processing. One service could invoke another service
for certain functionality by communicating through SOAP
messages. In the VolumeRendering application from
the previous section, the service for constructing a spatial-
temporal tree would invoke a service specified in creating
a temporal tree, since each leaf node in the spatial tree
presents its temporal information as a tree structure.
In order to facilitate the implementation of autonomic
properties for service components, our middleware includes
a Service Deployment Module, a Performance Module and
a Learning Module. When an application is submitted to
the system with its source code and the configuration file,
the Service Deployment Module first decomposes the ap-
plication and wraps it into autonomic service components.
This is done by AutoServiceWrapper, which will be dis-
cussed in the following subsection. Then, the configurator
extracts information such as dependencies between services
and input/output specifications for service communication.
The purpose of this module is to activate the application
processing by executing its autonomic service components.
Each service component is programmed with adjustable
serviceparameters,suchthatthedifferentchoiceofparame-
ters could significantly impact the execution time, process-
ing accuracy and the relative workload of different services.
As we stated earlier, adapting these service parameters to
maximize the benefit function within the time constraints, is
the key function of the middleware. We implement an auto-
nomic adaptation algorithm as part of the Learning Module.
To be specific, it records the historical data and trains the
system model and estimators during the normal processing.
Details of the algorithm will be discussed in Sect. 4. The
Performance Module cooperates with the Learning Module
by analyzing the benefit function to decide the priority of
each service component. This is important, as service pa-
rameters from a service with higher priority could have a
more significant impact on achieving our goal. Furthermore,
when a particular event is detected, the Learning Module ap-
plies the trained model and estimators to control parameter
adaptation. At this time, the Performance Module iteratively
queries the status of processing to avoid the violation of time
constraints specified in the computation task.
3.2 AutoServiceWrapper: an autonomic service component
We now describe how AutoServiceWrapper enables the cre-
ation of autonomic service component in our middleware.
Firstly, in addition to the traditional service interface, an au-
tonomic service component enhances its service description
to export information about its performance, interactivity,
and adaptability. We use the Web Services Resource Frame-
work (WSRF) from GT4.0 and claim the adjustable service
parameters as the stateful resources, since the value of the
Fig. 2 The detailed design of AutoServiceWrapper
parameter from the previous time step would be retrieved
for calculating the parameter value at the current time step.
Secondly, static analysis on the source code is performed
to insert checkpoints with the goal of monitoring processing
progress. To be specific, for each service parameter,a check-
point will be inserted after the loop, function call, or state-
mentswherethisparameterisinvolved.Further,wealsoran-
domly insert checkpoints in the rest of the code. In this way,
the processing is monitored at a micro level to capture the
essential relationship between the parameter values and the
execution time. Finally, the service description and instru-
mented service code are combined. The newly created ser-
vice will be registered in the service directory and is ready to
be invoked. This wrapping procedure is graphically shown
in Fig. 2.
The AutoServiceWrapper enables the autonomic proper-
tiesoftheservicecomponentbyassigninganAgent/Control-
ler object. During the normal processing, the agent executes
the autonomic adaptation algorithm and tries to estimate the
relationship between service parameters and the execution
time (T), relative workload (R), and the overall application
benefit (B). Furthermore, agents for different service com-
ponents also cooperate with each other in order to find out
the global pattern (macro effect) that will better improve the
system model. During the time-critical event handling, the
controller is activated. By using the learned estimators and
improved model from the agent, it simply controls how the
service parameters will be adapted to optimize the bene-
fit function before time expires. Together, the wrapping of
the service and the Agent/Controller object allow autonomic
service components to configure, manage, adapt, and opti-
mize their execution.
4 Autonomic adaptation algorithm
In this section, we present the adaptation algorithm imple-
mented in the Learning Module of our middleware.

Page 5

Cluster Comput (2009) 12: 87–10091
An application our middleware supports could comprise
of a series of interacting services. Currently, each service is
assumed to be deployed on a single node. The application
initiates one initial service, which then initiates, directly or
indirectly, all other invoked services. The workload of such
invoked service comprises of a series of function calls from
their invoking service. We assume that we have at least as
many nodes available as the services, and a separate node
could be used for each service. In the future, we will con-
sider the possibility that each service could be run on paral-
lel platforms.
In such a case, we have three considerations. Obviously,
we need to meet a time constraint, and second, we need to
maximize the benefit function. However, because of the in-
teraction between the services, we need to consider another
factor, which is the relative workload of each service. The
operating dynamics of a processor P executing one of the
services can be captured by the simple queuing model. At
time step k, λ(k) and β(k) denote the average arrival and
processing rates associated with the processor P. The input
queue size q(k) indicates the current workload on P, which
can be calculated as:
q(k) = (λ(k)−β(k))×Ts
where Tsdenotes the sampling period.
(3)
4.1 Algorithm overview
The intuition for our approach is as follows. The adaptation
process in our framework corresponds to a typical optimal
control model [12]. The objective of optimal control theory
is to determine the control signals that will cause a process
to satisfy the physical constraints,and at the same time,min-
imize (or maximize) some performance criterion. The input
to our middleware is a number of services with dependen-
cies involved. Each of them is programmed with adjustable
service parameters, as we had previously shown in our ex-
amples from Sect. 2. When a particular event occurs, our
algorithm controls the adaptation of the service parameters
so as to optimize the benefit function, while satisfying the
physical constraints, which are the pre-defined time interval
and relative workload of the services. We assume that our
processing comprises a series of processing rounds, and val-
ues of all service parameters can be modified between two
processing rounds.
Our approach is to first build a model for a particular ap-
plication and estimate the effect of service parameters on ex-
ecution time, relative workload of different services, and the
benefit function. Then, this model is deployed in the adapta-
tion process of the application. In the VolumeRendering
application, the model specifies how error tolerance and im-
age size would be changed based on their values from the
previous time step and the current action (increase/decrease
and by how much). However, the control problem of adjust-
ing service parameters is a high-dimensional continuous-
state task. For example, error tolerance is a real number be-
tween0and1.Thenits controlsignalcouldbeanyrealnum-
ber as long as the new value remains in the range. It is ex-
tremely difficult to build an accurate model from the begin-
ning. We propose an autonomic adaption algorithm that is
able to quickly learn to perform well in the parameter adap-
tion. Furthermore, the adaptation action taken at each ser-
vice component will be converted to an effect in the overall
system.Suchglobalpatterncouldbethecorrelationbetween
error tolerance and image size in our example. Once it is de-
tected, we could adapt both parameters in a non-conflicting
way.
The algorithm falls into two phases, namely, a normal
execution phase and a event handling phase. In the normal
execution phase, we locally explore different values of ser-
vice parameters to train the model and to estimate the re-
lationships between service parameters and execution time,
relative workload of different services, and the benefit func-
tion. Once an event is detected and needs to be handled with
a time constraint, the service parameters will be adapted us-
ing the learned model.
4.2 Detailed problem formulation
In order to utilize the optimal control theory to model the
adaptation process in our framework, we need to define a
system model. The variables we use in our model are listed
in Table 1. In the VolumeRendering example, the vec-
tor of state variables x(k) could be [error tolerance, image
size]T. A possible control signal could be [0.005,−20]T,
which increases error tolerance by 0.005 and decreases im-
age size by 20. Given the current values of these two para-
meters, we can estimate the overall execution time and rel-
ative workload of different services at time step k, which
is denoted as w(k) and q(k), respectively. Furthermore, the
corrective term v(k) may have to be applied because of the
possible correlation between error tolerance and image size.
State equation
time-varying discrete-time state equation:
The system then is described by the linear
x(k) = Ax(k −1)+Bu(k)+v(k)
(4)
Table 1 Definitions for the system model
VariableDescription
x(k)
u(k)
Adjustable service parameters
Increase/decrease to parameters
Estimated overall response time
Corrective term
Relative workload
w(k)
v(k)
q(k)