A method for efficient execution of bioinformatics workflows.

Abstract

Efficient execution of data-intensive workflows has been playing an important role in bioinformatics as the amount of data has been rapidly increasing. The execution of such workflows must take into account the volume and pattern of communication. When orchestrating data-centric workflows, a centralized workflow engine can become a bottleneck to performance. To cope with the bottleneck, a hybrid approach with choreography for data management of workflows is proposed. However, when a workflow includes many repetitive operations, the approach might not gain good performance because of the overheads of its additional mechanism. This paper presents and evaluates an improvement of the hybrid approach for managing a large amount of data. The performance of the proposed method is demonstrated by measuring execution times of example workflows.

Full text

Page 1

139
A METHOD FOR EFFICIENT EXECUTION OF
BIOINFORMATICS WORKFLOWS
JUNYA SEO1
j-seo@ist.osaka-u.ac.jp
YOSHIYUKI KIDO2
y-kido@ist.osaka-u.ac.jp
SHIGETO SENO1
senoo@ist.osaka-u.ac.jp
YOICHI TAKENAKA1
takenaka@ist.osaka-u.ac.jp
HIDEO MATSUDA1
matsuda@ist.osaka-u.ac.jp
1Department of Bioinformatic Engineering, Graduate School of Information Science
and Technology, Osaka University, 1-5 Yamadaoka, Suita, Osaka 565-0871, Japan
2The Center for Advanced Medical Engineering and Informatics, Osaka University,
2-2 Yamadaoka, Suita, Osaka 565-0871, Japan
Efficient execution of data-intensive workflows has been playing an important role in
bioinformatics as the amount of data has been rapidly increasing. The execution of such
workflows must take into account the volume and pattern of communication. When
orchestrating data-centric workflows, a centralized workflow engine can become a bottle-
neck to performance. To cope with the bottleneck, a hybrid approach with choreography
for data management of workflows is proposed. However, when a workflow includes many
repetitive operations, the approach might not gain good performance because of the over-
heads of its additional mechanism. This paper presents and evaluates an improvement
of the hybrid approach for managing a large amount of data. The performance of the
proposed method is demonstrated by measuring execution times of example workflows.
Keywords: web service, workflow, data transfer, database.
1. Introduction
Efficiently executing large-scale and data-intensive workflows common to scientific
applications must take into account the volume and pattern of communication.
Traditionally, two common architectures for implementing workflow are proposed.
Those are centralized orchestration and choreography [3].
Figure 1 shows the two traditional architectures. Centralized orchestration de-
scribes how services can interact at the message level, with an explicit definition
of the control and data flows. This architecture has a workflow engine to execute
workflows. The engine acts as a controller of the involved services. Both control
and data flow messages are passed through this central engine. Examples of such
engines in bioinformatics domain are Taverna [6] and Triana [8].
On the other hand, a choreography model describes a peer-to-peer collaboration
between collections of services. This architecture basically does not have a central-
ized engine that controls workflows. Choreography focuses on message exchange, all
involved services are aware of their partners and when to invoke operations. The

Page 2

140 J. Seo et al.
??????????????????
??????????????
??????? ???
??????????????????? ????????????? ?!?
"$#&%?' ()?* +,#?-
.?( /102#43'()?'+ 56%
????????????????!??????????????????? ?????7????????????
"?0256(#?5?8?( )?9&0?:
;
)?' )=<1*5&> ?@56( ACB*5?>D"&5?%?'(56*
Fig. 1.Traditional architectures for workflow execution.
??????????????? ???
??????????????
??????????
??????? ??????? ??????????????!????"?
#%$?&?'()
*?+-,/.%0*/+-,/.%0*?+-,/.10
2%354376!89?:;<9
'=->
89?:A@?9?B!4
'
9?8
Fig. 2.Hybrid architecture of centralized orchestration and choreography.
Web Services Choreography Description Language (WS-CDL) [4] is an XML-based
language for describing this model.
Centralized control is suitable for scientific workflows because those workflows
contain many services with complicated controls (such as conditional branches).
However, in a centralized engine, every resulting output of each service is sent back
to the engine. Thus the engine can easily become a performance bottleneck by the
concentration of returning data. While a choreography architecture can optimize
data transport between services, but the implementation of this architecture is
highly complex and rigid.
Baker et al. proposes a hybrid architecture that combines an orchestration with
a centralized controller (a workflow engine) and a choreography model of distributed
data transport [2]. Figure 2 shows an image of the architecture. The architecture
mixes benefits of the above two architectures. It introduces proxies between Web
services and a workflow engine. The engine sends a request to a proxy instead of
a service. The proxy invokes the service and save its result to a local storage, and
returns a unique ID of the result to the engine. The result (intermediate data) is
not sent back to the engine but is transferred directly to other proxies according
to the data flow in the workflow. Therefore the engine does not have to receive the
intermediate data to be sent to the next service.
In this paper, we propose a modification of the hybrid architecture. The hybrid

Page 3

Efficient Execution of Bioinformatics Workflows141
?????????
??????????????????
?????????
??????????????????
?????
???"!#?
$???%
??&#'(?
)%
?+*?'(,&
-
???
?+?"!??
.
??%
??&#'(?
)?0/
1??32
??4

*65????
-?-
???
???"!#?
-
???7?
?+?8!#?
-
$??7%
??&#'(?
)%
??*?'9,&
-;:
?
??4??+?
-?.
??%
??&#'(?
<?/
=???/
??, ??+???
-?>
?A@3B8C
D
?
?;&64+E??
F(?A@3B8C
:
?
??4??+?
???8?G!?5H,??I
J

E??

?
Fig. 3. Execution steps of a workflow with two Web services in the original hybrid architecture.
architecture can suppress the amount of data transfer by removal of the transfer
to/from the engine. Thus its performance is highly improved when the amount of
the resulting output of each service is very large. However, some overheads exist in
the hybrid architecture at the proxies. When it is adopted to a workflow including
repetitive operations (such as loop operations), the engine need to frequently inter-
act with proxies for invoking services and transferring their results to other proxies.
We change the protocol between the engine and proxies for reducing the overheads
as described in Sec. 3.
2. Workflow Operations in Hybrid Architecture
Workflows are frequently used in various scientific fields. Especially, in the bioinfor-
matics domain, workflows can be used for integrating a number of remote resources
in terms of data and applications [1]. Many analysis tools are available as Web
services from EBI [5, 9], DDBJ [10], and KEGG [11]. Scientists often use several
databases for their analyses and compose workflows with several Web services that
are distributed geographically.
Figure 3 shows the execution steps in a workflow containing two Web services
in the hybrid architecture [2]. Those steps are described as follows in detail.
1: A workflow engine invokes a proxy to make an invocation to a Web service
(Fig. 3 left).
2: The proxy invokes the Web service.
3-4: The proxy receives the result of the service, and save it within the proxy.
There is a requirement that the proxy has enough disk space to store the
result.
5: The engine receives only the ID of the result from the proxy instead of its
whole data.

Page 4

142 J. Seo et al.
?????????
??????????????????
??????????????
??????? ???
???????!?
?"???#?$???%??????
&?')(
???????*?
+?'),
??-*./???
,
??0?./? -
1
')(
???????!?
2
'$,
??-*./???
(3
4
'!5
??6???07?%???
8?')(
?$?????*?
9
':(
???????*?
;/':,
?$-*./???
,
??0?./?-
<?'
??6????
1>=
':,
??-*./???
(3
Fig. 4. Modified execution steps of a workflow in the proposed hybrid architecture.
6: The engine invokes the proxy again to make transfer the result (the operation
is named deliver in Fig. 3).
7: The proxy invokes another proxy (Fig. 3 right) to notify data transfer.
8: The second proxy receives the result from the first proxy (the operation is
named stage in Fig. 3).
9-10: After the data transfer finished, the second proxy returns acknowledgment
to the first proxy, and the first one also returns it to the engine.
11-15: The engine invokes the second proxy with the ID, and the second proxy
invokes another Web service.
As shown in Fig. 3, additional invocation steps are necessary compared with
a centralized orchestration (2, 6, 7, and 12). If the workflow includes repetitive
operations, those extra invocations are carried out at the number of repeats. The
extra invocations may increase the processing time of the workflow.
3. Improved Method
As the amount of biological databases has been increasing rapidly, the data to
be managed in workflows has also been increasing. Those databases contain huge
numbers of entries. For example, there are over 9 million entries are registered in the
UniprotKB database [12]. When scientists want to analyze such a huge number of
data in their workflows with Web services, repetitive invocations of the services are
occurred. If they use several databases in combination, the total number of resulting
data should be very large. Those repetitive invocations in a workflow cause large
overheads in the hybrid architecture.
To decrease the number of the extra operations described in the previous section,
we present a modification to change the execution steps in the hybrid architecture.
The modified execution steps are described as the followings (see Fig. 4).

Page 5

Efficient Execution of Bioinformatics Workflows 143
1: A workflow engine invokes a proxy to make an invocation to a Web service
(Fig. 4 left).
2: The proxy invokes the Web service.
3-4: The proxy receives the result of the service, and transfers it to another proxy
(Fig. 4 right).
5: The engine receives only the ID of the result from the first proxy instead of
its whole data.
6-11: The engine invokes the second proxy with the ID, and the second proxy
invokes its Web service. If it is not necessary to transfer the result of the
Web service to other proxies, the proxy saves the result within the proxy.
In the proposed modification of the execution steps, we decrease overhead at the
following two points:
• Transfer the result of a Web service to the next proxy directly.
A proxy transfers the result to the next proxy directly. In this case, the proxy
does not need to save the result within the proxy. Thus one step is removed
from previous hybrid architecture (Step 4 in Fig. 3).
• Remove deliver and stage operations.
By transferring the result directly, the workflow engine and the first proxy
does not need to invoke the first proxy and the second proxy, respectively,
to make transfer the result. Thus deliver and stage operations and receiving
their acknowledgments (Steps 6, 7, 9 and 10 in Fig. 3) can be removed.
Totally we can remove two Web service invocations and one data saving within
the first proxy. The proxy we have proposed is deployed as a Web service as the
same as the original hybrid architecture. Figure 5 shows the deployed operation in
our proxy. Our proxy has only one operation to invoke a Web service and to make
data transfer.
The operation “invoke” receives 4 arguments. “wsdl”, “operation” and “args”
are the same as those in the original hybrid architecture. There is a difference at
argument “returns”. Our operation can handle multiple results so that the workflow
engine can extract resulting data to be returned. The destination information of the
data transfer is also included in “returns” like:
resultA->hybrid.ist.osaka-u.ac.jp
resultB->http://hybrid.ist.osaka-u.ac.jp:8080/axis2/services/Hybrid?wsdl
Users can specify transferring destination as its host name or WSDL location.
Users also can specify different destinations in each result.