Conference PaperPDF Available

Regression Testing Web Services-based Applications

Authors:

Abstract and Figures

Web applications can be composed of heterogeneous selfcontained web services. Such applications are usually modified to fix errors or to enhance their functionality. After modifications, regression testing is essential to ensure that modifications do not lead to adverse effects. In this paper, we present a safe regression testing algorithm that selects an adequate number of non-redundant test sequences aiming to find modification-related errors. In our technique, a web application and the behavior of its composed components are specified by a two-level abstract model represented as a Timed Labeled Transition System. Our algorithm selects every test sequence that corresponds to a different behavior in the modified system. We discuss three situations for applying this algorithm: (1) connecting to a newly established web service that fulfills a composed web service, (2) adding or removing an operation in any of the composed web services, (3) modifying the specification of the web application. Moreover, modifications handled by the algorithm are classified into three classes: (a) adding an operation, (b) deleting an operation, (c) fixing a condition or an action. Key-words : label transition systems, testing, verification, web service, web application.
Content may be subject to copyright.
Regression Testing Web Services-based Applications
Abbas Tarhini (1) Hacène Fouchal (2) Nashat Mansour (3)
(1) LICA/CReSTIC, Université de Reims Champagne-Ardenne Moulin de la Housse,
BP 1039, 51687 Reims Cedex 2, France. E-mail:Abbas.Tarhini@univ-reims.fr
(2) GRIMAAG, Université des Antilles et de Guyane, F-97157 Pointe-à-Pitre,
Guadeloupe, France. E-mail:Hacene.Fouchal@univ-ag.fr
(3) Computer Science and Mathematics Division, Lebanese American University,
PO Box 13-5053, Beirut, Lebanon. E-mail: nmansour@lau.edu.lb
Abstract
Web applications can be composed of heterogeneous self-
contained web services. Such applications are usually mod-
ified to fix errors or to enhance their functionality. After
modifications, regression testing is essential to ensure that
modifications do not lead to adverse effects. In this paper,
we present a safe regression testing algorithm that selects an
adequate number of non-redundant test sequences aiming
to find modification-related errors. In our technique, a web
application and the behavior of its composed components
are specified by a two-level abstract model represented as
a Timed Labeled Transition System. Our algorithm selects
every test sequence that corresponds to a different behavior
in the modified system. We discuss three situations for ap-
plying this algorithm: (1) connecting to a newly established
web service that fulfills a composed web service, (2) adding
or removing an operation in any of the composed web ser-
vices, (3) modifying the specification of the web applica-
tion. Moreover, modifications handled by the algorithm are
classified into three classes: (a) adding an operation, (b)
deleting an operation, (c) fixing a condition or an action.
Key-words : label transition systems, testing, verification,
web service, web application.
1 Introduction
The development of web applications has received signif-
icant attention in the past few years. They have been re-
markably introduced into all areas of communication, in-
formation distribution, e-commerce and many other fields.
The use of web services also provided a common commu-
nication infrastructure to communicate through the internet,
and enabled developers to design applications that can span
different operating systems, hardware platforms and geo-
graphical locations. Thus the design and the maintenance of
reliable web applications and web services should be con-
sidered seriously.
Originally, web sites were constructed from a collec-
tion of web pages containing text documents and intercon-
nected via hyper links. Recently, the dramatic evolution
of web technology has led to web applications that can
be built by integrating different components from variety
of sources, residing on distributed hardware platforms, and
running concurrently on heterogeneous networks. The con-
struction of systems from different types of software com-
ponents faces various challenges such as maintaining per-
formance, reliability and availability of those systems. But
the validation of such web applications remains a major
challenge. A Web application might invoke multiple web
services located on different servers with no design, source
code or interface available. This forces designers to use
black-box notions to select relevant web services from the
pool of services found on the internet. With the increasing
number of periodic publishing of web services, web appli-
cations need more often to be updated so that they select the
most optimal and reliable service. Moreover, web systems
are usually exposed to structural changes and modifications.
These changes require us to retest the web application in or-
der to provide confidence that the system functionality and
unmodified parts have not been adversely affected by the
modifications. Regression testing refers to selecting tests
from the test suite generated during the initial development
phase and to adding new tests to address enhancements and
additions. One regression testing strategy is retest all which
reruns every test in the initial test suite. This approach is
normally very expensive and requires a lot of time. An alter-
native approach is to select a random subset of tests, which
might be unreliable. Therefore, regression testing is a chal-
lenging task that should be both economic and reliable.
In this paper, we present a safe regression testing tech-
nique that is used to retest the web system whenever it
is modified. First, we suggest a two-level abstract model
to represent a web application. The generation of test se-
quences and test histories for the initial development phase
is described in previous work [1]. Next, we present a re-
gression testing technique that retests the modified web sys-
tem with only the necessary test sequences selected from
the test histories and provide confidence that modifications
(including additions or deletions) are correct and have not
adversely affected other parts in the system. In this work
we do not deal with performance, we only target correct be-
havior. Moreover, the state explosion problem is handled by
using the hierarchical two-level abstract model that repre-
sents the web application. With such a model, information
handled by the tester at the first level of abstraction is dif-
ferent from that handled at the second level.
The rest of this paper is organized as follows. In Section
2 we present a brief background on web service models and
our previous work done on testing web services. The model-
ing of web applications and needed definitions are presented
in Section 3. In Section 4 we present a regression testing al-
gorithm. We conclude the paper in Section 5.
2 Background
In this section, we present overviews of web services infras-
tructure and of regression testing.
2.1 Web Services Overview
Web Services were defined differently by vendors, re-
searchers, or standards organizations. In our study, we de-
fine Web Services as self-contained component-based ap-
plications, residing on different servers, and communicat-
ing with other applications by exchanging XML messages
wrapped in SOAP interfaces.
Web services infrastructure is based on service-oriented
architecture (SOA) that involves three kinds of participants:
service providers, service requesters and a service broker
(Figure 1). A service provider publishes services to a ser-
vice broker. Service requesters find required services us-
ing a service broker and then bind to them [2]. This in-
frastructure uses the following standards to make web ser-
vices function together: Web Services Description Lan-
guage (WSDL), Universal Description, Discovery and Inte-
gration (UDDI), The Extensible Markup Language (XML),
and Simple Object Access Protocol (SOAP).
Service Broker
Find
Bind
Publish
Service provider Service requester
Figure 1: Web service architecture
After creating a web service, the service provider gen-
erates the corresponding WSDL file and publishes it on the
internet. The WSDL file is a description of how to ac-
cess the web service and what operations this service can
perform. On the service broker, the UDDI registry holds
the specification of services and the URL that points to the
WSDL file of services. The service requester searches for
a web service in the UDDI registry, then binds to it, and
transmits massages and data using XML wrapped in SOAP
interfaces.
2.2 Regression Testing Overview
Regression Testing is a process which tests a system once
again to ensure that it still functions as expected by spec-
ification [3]. The reason for this renewed testing activity
is usually performed when changes are done to a system
ωproducing a modified version ω. Regression testing is
a way to test the modified version ωusing a test set T
used previously to test the original system ω. The selec-
tion of suitable test cases from Tcan be made in differ-
ent ways and a number of regression-testing methods have
been proposed. These methods are based on different ob-
jectives and techniques, such as: procedure and class fire-
walls [4, 5]; semantic differencing [6]; textual differencing
[7]; slicing-based data-flow technique [8, 9]; test case re-
duction [10, 11]; and safe algorithm based on program’s
control graph [12]. Typical regression testing procedures
follow five main steps: (1) Identify the modifications made
to ω; (2) test selection step: using the results in step 1, select
TT, a set of tests that may reveal modification-related
errors in ω; (3) if necessary, create new tests for ωand ap-
pend to T. These may include new functional tests required
by changes in specifications, and/or new structural tests re-
quired by applicable coverage criteria; (4)run Ton ω, to
provide confidence about ω’s correctness with respect to
T; and finally, (5) create T′′, a new test set/history for ω.
Further, test histories should be maintained. The system’s
test history identifies, for each test, its input, output and ex-
ecution history. An execution history consists of a list of
components and their internal states exercised by the test.
Moreover, [12] emphasized the creation of a test history for
the test suite so that a regression test can be applied.
Rothermel and Harrold [12], presented a safe algorithm
for regression test selection. It constructs control depen-
dence graphs (CDG) for the original and modified program,
then uses these graphs to generate tests.In fact, this algo-
rithm selects all existing tests in a region when a simple
change is done at the top level of the program. More-
over, this method does not identify parts of the program that
should be covered by tests. These drawbacks were handled
by the authors in a successive work Elaborating more on this
algorithm, Rothermel and Harrold also implements and con-
ducted empirical studies on several subject programs and
modified versions [13]. The results suggests that in practice,
the algorithms can significantly reduce the cost of regression
testing a modified program.
2
Research on regression testing has focused on struc-
tured and object-oriented software and it is lacking for web
services based software applications. In this work, we are
concerned with the five steps of the regression testing ap-
proach mentioned before. Our regression testing algorithm
selects test cases that identify modifications concerning the
addition and deletions of web services to the system, and
changes in the specification of the web system or any of its
components. It also creates new functional tests required by
changes in the specifications. Moreover, the algorithm up-
dates the test history so that it maintains an acceptable test
suite size.
3 Modeling Web Applications
Web-based software systems are constructed by integrating
different interacting-components from a variety of sources.
The schedule of invoking the interacting-components is re-
stricted by the requirements specification of the web appli-
cation and by time constraints. These components interact
with the main application as well with other components
by exchanging messages (actions) that might also involve
timing constraints. To model such systems, we suggest a
two-level abstract model. The first level models the interac-
tion of components with the main application. The second
level models the internal behavior of each component in the
system. This hierarchical model helps in minimizing the
state explosion problem. In the following subsections we
described each model and illustrate it with examples.
3.1 Web Application Representation
The functional behavior of a web system could be repre-
sented as a Task Precedence Graph (TPG). However, since
we study web applications composed of components that
interact by exchanging messages restricted by timing con-
straints, our first level of abstraction models a web applica-
tions as an input-complete Timed Labeled Transition Sys-
tem (TLTS), where each node in the TLTS is an abstract rep-
resentation of a single component in the system that models
the behavior of its software modules, and an edge joining
two nodes represents the flow of actions (transitions) be-
tween components. Every edge is labeled with an action
and its corresponding timing constraint. We formally define
an TLTS as follows:
Definition 3.1 (Timed Labeled Transition System (TLTS))
An TLTS is defined by M= (S, A, C, T , s0)where Sis a
finite set of states, s0is the initial state, and Ais a set of
actions. Ais partitioned into 2 sets: AIis the set of input
actions (written ?i), AOis the set of output actions (written
!o). Cis a set of clocks.
Tis a transition set having the form {T r1.T r2...T rn};T ri
= <s; a; d; EC; Cs>, where: sSand dS are starting
and destination states; aAis the action of the transition;
EC is an enabling condition evaluated to the result of the
formula abwhere ∼∈ { <, >, ,, = } or to a constant
valued either true or false; Csis a set of clocks to be reset at
the execution of transition T ri.
Definition 3.2 (input-complete) ATLTS Mis said to be
input-complete if all states accept any input aAI. A TLTS
will be input-completed by adding to each controllable state
(a state whose action is only input) a loop labeled by all
complementary actions in the input alphabet AI.
MC
¹¸
º·
-
CR
¹¸
º·
HR
¹¸
º·
W P
¹¸
º·
¸
?Car Rental;c=0
®
!Notice; c<10
R
?Hotel Reservation;c=0
I
!Notice; c<10
¾?Weather Request
-
!weather info
Figure 2: An example of TLTS representing a simple travel
agency web application
Figure 2 illustrates a TLTS representing a simple travel
agency web application that is composed of four compo-
nents: Main Component (MC), Hotel Reservation (HR), Car
Rental (CR), and Weather Prediction (WP).
3.2 Single Component Representation
The second level of abstraction models every single compo-
nent in the web application. In this level, we model each
component as an input-complete Timed Labeled Transition
System (TLTS). Each state in the TLTS represents a state of
the modeled component. An edge joining two states is la-
beled with an action and its corresponding timing constraint.
It represents a transition from one state to another.
Figure 3 shows an example of TLTS representing a sim-
ple hotel reservation component (HR)with initial state s0.
A transition is represented by an arrow between two states
and labeled by the action, the timing constraint and clocks
to reset (action; EC; Cs). The TLTS in figure 3 is input-
complete because it accepts any input event in the input al-
phabet A; that is, if at state s0the user input an invalid date
the system accepts the input but it stays in s0; otherwise, it
moves to state s1where the users may choose either a sin-
gle or a double room, thus, the system may move to either
state s2or s3. As soon as the appropriate input is selected
the corresponding price is given, and clock cis set to zero
in order to count the time for the conformation back from
3
s0
¹¸
º·
-s1
¹¸
º·
s2
¹¸
º·
s3
¹¸
º·
s4
¹¸
º·
s5
¹¸
º· s6
¹¸
º·
µ´
¶³
-
?valid date in/out
À
?invalid date
?
?single
À
?wrong input
-
?double
Y
!Not Available ¦
¦
¦
¦
¦²
!Price ; c=0
O
!Not Available
¡
¡
¡
¡µ
!Price ; c=0 CCCW
?Confirmation ; c<5
-
!Notice:Conf/Price
Figure 3: A TLTS representing simple hotel reservation
the user, then, the system moves to state s4. If the confor-
mation is not sent with in the time (c<5) the session will be
timed-out.
4 Regression Testing
Our regression testing technique for web services is based
on a previous work [1] on testing web services. In the fol-
lowing subsections we briefly discuss our previous work on
testing web services. Then, we present the regression test
case selection algorithm.
4.1 Testing Web Services
In previous work [1], we presented testing technique that
guarantees the availability of services in our web applica-
tion. It selects and then associates all suitable web services
to our web application before invocation time; moreover, it
suggests testing the functionality of the web service inte-
grated in the web application by executing three sets of test
cases generated from (1) the WSDL files and (2) the spec-
ification of both the component fulfilled by a web service
and (3) the specification of the whole web application. The
links to all selected suitable web services are saved into a
log file
associated with the component to be fulfilled by the
web service. The log file contains the urls of all suitable web
services and the set of test sequences used to test this com-
ponent, it also contains a priority ranking number for each
of these services. Using this log file, the web application
would have a wide range of finding available and suitable
web services at invocation time; further, this log file will
be essentially needed for retesting this component after any
modification. This method tests the web service individu-
ally (as a stand-alone component) and as a part of the web
CBS.
The first set tests the adequacy of the web service inde-
pendently. It is generated based on boundary value testing
analysis [14] bounded by the limitations defined in the XML
schema. The second set of test sequences is used to test the
behavior of the web service individually. Thus, we gener-
ate this set by traversing all loop-free paths going from the
initial state of the TLTS specification representing the com-
ponent to be fulfilled. The third set of test sequences tests
the interaction of the web service as a part of the web ap-
plication. Thus, we generate this set by traversing all loop-
free paths going from the initial state of the TLTS represent-
ing the web application including the loop-free paths of the
TLTS representing the inner actions of the composed com-
ponents. Next, test sequences and test histories are created.
One test history is created for the web system; it consists
of the third set of test sequences generated above, and an
execution history for each test sequence. An execution his-
tory consists of a list of components and their internal states
that experienced this test sequence. Other test histories are
created for each composed component. Those test histo-
ries are attached to log files found in their corresponding
component. A component’s test history consists of only test
sequences that experienced this component. After briefing
our previous work, we will present the new work done on
regression test case selection in the following sub sections.
4.2 Regression test case generation
Web application are subject to modifications. Modifications
to one or more components might affect other components
of an application, which might lead to errors. We classify
modifications to web service based applications into the fol-
lowing types: (a) Type-1: integrating a newly established
web service into the application, (b) Type-2: adding, re-
moving or fixing an operation or a timing constraint in an
existing component, (c) Type-3: modifying the specification
(operations or timing constraints) of the web application.
When modifications occur, we retest the modified system to
gain confidence about its correctness. A TLTS for the mod-
ified version is constructed. It reflects all additions and/or
deletions of states or edges performed by the modification.
Each of these modifications together with needed examples
are described in sections 4.2.2, 4.2.3, and 4.2.4.
Note that the second and third types of modifications
are reflected in their corresponding TLTSs by adding and/or
removing states and edges from that TLTS depending on
the modification performed. Therefore, the modified TLTS
should be input-completed (see Definition 3.2) so that it will
not have any dangling edges in case of deletions; dangling
edges does not allow generating executable test sequences.
Consider a web application ωand its modified version
ω. After any of the above modifications, we need to vali-
date ωby using the set of test sequences Tused previously
to test the web application ω. Thus, we firstly identify all
modifications done to ω, then select a set of tests TT
that may reveal modification-related errors in ω. Moreover,
a new test set may be created to test required changes in the
specification of the web application or any of its composed
4
components. The algorithm for selecting the test set Tis
shown in section 4.2.1.
4.2.1 Test Selection Algorithm
In this section we present a safe regression testing algorithm
that selects the necessary test sequences believed to find
modification-related errors. This algorithm is used for
Type-2 modification, which includes (a) fixing a time
condition or action, (b) removing an operation, or (c)
adding an operation. It is also used for part of Type-3
modification. The algorithm is presented in Figure 4.
Algorithm: TestSelect
Input: TLTS for ω, Test set T. Test history.
Output: Test set T. Updated Test history.
Step 1: Complete the TLTS for ωto satisfy the Definition 3.2.
Step 2: Generate Test set T
all for ωby traversing all acyclic
paths of TLTS ωfrom the initial state.
Step 3: Generate Tand update the test history as follows:
All test sequences found in T
all and not found in Tare
executed, that is, add to the retest set T.
Thus T=T
all \T.
All test sequences found in Tand not found in T
all are
deleted from the test history.
All test sequences found in both Tand T
all are kept in
the test history but not re-executed.
Add test sequences in Tto the test history.
Figure 4: Algorithm to generate test set T.
The input to the TestSelect algorithm presented in Fig-
ure 4 is the modified TLTS version for ω, the previously
generated test set Tfor ω, and the test history. The output
is a selected set of test sequences TTthat is believed
to reveal modified-related errors if executed on ω, and the
updated test history.
Given the input-complete TLTS for ω, we get the modi-
fied TLTS version for ωby adding to (and/or deleting from)
ωat least, (1) a new state si, and/or (2) an edge (si,sj).
The TLTS for ωshould be completed to satisfy the input-
completeness definition (see Definition 3.2). In the exam-
ples below, we show why completing a TLTS is very im-
portant for T estSel ect algorithm to operate correctly espe-
cially if the modification is a deletion.
The set Tmust include all test sequences that (1) traverse
the newly added state in TLTS ω, (2) traverse all newly
added edges in TLTS ω, and (3) traverse all edges with
modified labels in the TLTS ω. Thus, Tis generated by
finding all acyclic paths in the input-complete TLTS of ω
and not in the input-complete TLTS of ω. The test set T
is able to experience any of the changes (fixing, deleting or
adding of a timing constraint or an action) in the Type-2 and
Type-3 modifications presented in Sections 4.2.3, and 4.2.4.
To illustrate, we consider each of these changes separately:
(a) Fixing a condition or an action:
Consider the TLTS ωin figure 5. Assume the time re-
striction in the transition <!Notice;c<10;-> is changed
to (c<5). thus, we get a modified version ωwith time
restriction <!Notice;c<5;->. The set Tgenerated from
the original TLTS ωis:
T1: <?Hotel Reservation;c=0;->
T2: <?Car Rental;c=0;->.<!Notice;c<10;->
T2: <?Weather Request;c=0;->.<!Weather Info;-;->
The set T
all generated from modified TLTS ωis:
T
all1: <?Hotel Reservation;c=0;->
T
all2: <?Car Rental;c=0;->.<!Notice;c<5;->
T
all3: <?Weather Request;c=0;->.<!Weather Info;-;->
Thus, the set T=Tall \Tis:
T
1: <?Car Rental;c=0;->.<!Notice;c<5;->
(b) Removing (deleting) an operation:
Consider the original TLTS ωin figure 2. By deleting
component HR we get the modified TLTS ωin figure
5. We input-complete ωby adding a transition loop
labeled (?HotelReservation;c= 0).
MC
¹¸
º·
-
CR
¹¸
º·
W P
¹¸
º·
¸
?Car Rental;c=0
®
!Notice; c<10
k?Hotel Reservation;c=0
¾?Weather Request
-
!weather info
Figure 5: A modified TLTS for the original travel agency web
application presented in figure 2
The set Tgenerated from the original TLTS ωis:
T1: <?Hotel Reservation;c=0;->.<!Notice;c<10;->
T2: <?Car Rental;c=0;->.<!Notice;c<10;->
T3: <?Weather Request;c=0;->.<!Weather Info;-;->
The set T
all generated from modified TLTS ωis:
T
all1: <?Hotel Reservation;c=0;->
T
all2: <?Car Rental;c=0;->.<!Notice;c<10;->
T
all3: <?Weather Request;c=0;->.<!Weather Info;-;->
Thus, the set T=Tall \Tis:
T
1: <?Hotel Reservation;c=0;->
The test history will be updated as mentioned in the
algorithm, a part of the updated test history is:
T1: <?Hotel Reservation;c=0;->
T2: <?Car Rental;c=0;->.<!Notice;c<10;->
T2: <?Weather Request;c=0;->.<!Weather Info;-;-> . .
.
5
(c) Adding an operation:
Consider the original and modified web services shown
in figure 6. The set Tgenerated from the original TLTS
ωis:
T1: <?wrong input;-;->
T2: <?cityName;-;->.<!humidity;-;->.<!weather Info;-;->
The set T
all generated from modified TLTS ωis:
T
all1: <?wrong input;-;->
T
all2: <?cityName;-;->.<!humidity;-;->.<!weather Info;-;->
T
all3: <?countryCode;-;->.<!humidity;-;->.<!weather
Info;-;->
Thus, the set T=Tall \Tis: T
1: <?countryCode;-;-
>.<!humidity;-;->.<!weather Info;-;->
Based on the illustrations presented above, note that al-
gorithm "T estSel ect" selects every test sequences that tra-
verse only the modified parts in the system, and only non-
redundant test sequences are re-executed. Moreover, it cre-
ates new test sequences to test added parts in the TLTS. This
algorithm considers effects of additions, deletions as well as
modifications in the web application. Thus, this algorithm is
safe regarding the whole behavior, and it is precise since it
selects only test sequences that exhibit a different behavior.
Moreover, the two-level hierarchical model helps in re-
ducing the state explosion problem. In fact, by representing
realistic web applications with such a model, the number
of states to be covered in each TLTS will be very small;
thus, the information handled by testers at each level of ab-
straction is different from that handled at the other level.
Further, the generated test cases use only symbolic values
for variables satisfying an adequate coverage criteria at test
execution.
4.2.2 Modification type 1: connecting to new web ser-
vices
In this type, the specification of the web system and com-
posed components did not change; however, modifications
occur when web services fulfilling composed components
are updated; that is, a new web service is appended to other
services, having the same functionality, that fulfill a compo-
nent in the web system. Therefore, first we need to test this
web service as stand-alone and if operates correctly when
communicating with our component; further, we need to
make sure it operate correctly when integrated in the web
system. Thus, in this case regression testing is applied to
retest the behavior of the updated component only; that is,
selection of test cases should be only in the third set of test
sequence. The three sets of test sequences needed to retest
the modified web application ωare created as follows:
Generate the first set of test sequences from the corre-
sponding WSDL of that service as described in section
4.1.
The second set of test sequences is going to be used
again as is, since we need to test the behavior of the
newly added web service individually and when com-
municating with the updated component. This set is
selected using the test history saved in the log file as-
sociated with this component.
Using the test history of the web system, the third set
of test sequences is selected from the previously gen-
erated test suite Tof the web application. This set in-
cludes only test sequences that exercises the modified
component. To illustrate consider figure 2, for exam-
ple, assume a new web service for weather prediction
is found and fulfills the component (W P ). Using the
web system test history, the newly generated third set
consists of only one test sequence, as shown below,
starting from the main component (MC) and passing
through weather prediction component (W P ):
T’: <?WeatherRequest;-;->.<?cityName;-;->.<!humidity;-
;->.<!WeatherInfo; -;->
4.2.3 Modification type 2: modify a component’s spec-
ification
In this type, the specification of a composed component is
modified. A change can be either adding or removing an
operation in a composed component. For example, assume
a new operation that returns the weather information based
on country code is needed. Thus the specification represent-
ing the weather prediction component is changed to handle
the input ?countryC ode. The TLTS of the modified and
original component specification is shown in figure 6.
s0
¹¸
º·
-s1
¹¸
º·
s2
¹¸
º· s3
¹¸
º·
µ´
¶³ s4
¹¸
º·
-
?cityName
À
?wrong input
BBBBN
!humidity
?
?countyCode
-
!humidity -
!weather Info
s0
¹¸
º·
-s1
¹¸
º·
s2
¹¸
º·
µ´
¶³ s3
¹¸
º·
-
?cityName
À
?wrong input
?
!humidity
-
!weather Info
Figure 6: TLTS Original and Modified Weather Prediction web
service
6
The three sets of test sequences needed to retest the modified
web application ωare created as follows:
Generate first set of test sequences to test only the
newly added operation (method) of that service. This
is done by parsing the corresponding WSDL file of that
service to get the data types and boundaries of the input
parameters and output information. Thus, the above
WSDL file is parsed and new test cases are generated
only for operation searchByCode as described in sec-
tion 4.1.
The second set of test sequences is generated using the
algorithm T estSel ect presented in section 4.2.1. For
example, the second set of test sequences for weather
prediction web service shown in figure 6 will exercise
only the newly added path in the TLTS as shown be-
low:
T’: <?countryCode;-;->.<!humidity;-;->
The third set of test sequences is also generated us-
ing the algorithm T estSel ect presented in section
4.2.1. It consists of all test sequences that exer-
cise only the modified component composed in the
web application. For example, consider the web
application present in Figure 2 and the modified
weather prediction component presented in Figure 6,
the third set of test sequences, shown below, con-
sists only of test sequences traversing the path in-
voked with input ?W eatherRequest in the web ap-
plication and also traverses only the newly added
path invoked with ?countryCode in the composed
component: T’: <?WeatherRequest;-;->.<?countryCode;-
;->.<!humidity;-;->.<!Weather Info;-;->
4.2.4 Modification type 3: modify the web application
specification
In this type, the specification of the web application is mod-
ified. A change can be either adding or removing an oper-
ation/component in the web application. To illustrate, con-
sider the web application presented in Figure 2, assume a
new component "credit card validation" (Ccv) is added to
the web application so that the hotel reservation HR and car
rental CR components can use it to validate credit cards.
Figure 7 shows the TLTS of the modified web application
and the TLTS of the component C cv.
With this modification we need to coordinate between
the newly added web service Ccv and the components H R
and CR. This is done by searching the UDDI registry again
to find the perfect match that coordinates between composed
web services. The three sets of test sequences that make up
test suite Tneeded to test the modified web application are
generated as follows:
Generate first set of test sequences to test only the
newly added web service. This is done by parsing the
MC
¹¸
º·
-
CR
¹¸
º·
HR
¹¸
º·
W P
¹¸
º·
Ccv
¹¸
º·
¸
?Car Rental;c=0
®
!Notice; c<10
R
? HotelRes;c=0
I
!Notice; c<10
¾?Weather Request
-
!weather info
q
? CC type/number
¾!valid/notValid
6
? CC type/Nbr
°
!valid/notValid
s0
¹¸
º·
-
s1
¹¸
º·
s2
¹¸
º·
µ´
¶³
¿
¿
¿
¿
¿7
? CC type/Number
s
?wrong input
eeeeeR
! valid/notValid
Figure 7: TLTS of the modified travel agency application and
TLTS of the C cv component
corresponding WSDL of that service to get the data
types and boundaries of the input parameters and out-
put information as described in section 4.1.
The second set of test sequences is generated by
traversing all paths starting from the initial state of the
TLTS representing the newly added component. For
example, the second set of test sequences for credit
card validator web service generated from the TLTS
shown in figure 7 looks like:
T
1: <?wrong input;-;-> T
2: <?CC type/Number;-;-
>.<! valid/notValid;-;->
The third set of test sequences is generated using the al-
gorithm T estSel ect presented in section 4.2.1. It con-
sists of all paths in the web application’s TLTS that
traverse all components affected by the new modifi-
cation, including the inner paths of the TLTS repre-
senting those components. The TLTS of the composed
hotel reservation HR and Credit card validation Ccv
components is shown in figure 8. A sample of the third
set for the modified web application shown in figure 8
would be:
T’: <?HotelReservation;-; c=0><?valid date;-;->.<?double;-;-
>.<!Price;-;c=0>.<?Confirmation;c<5;->.
<?CC:type/Number;-;->.<! valid/notValid;-;->.<!Notice;-;
c<10>
7
s0
¹¸
º·
-s1
¹¸
º·
s2
¹¸
º·
s3
¹¸
º·
s4
¹¸
º· s5
¹¸
º· s6
¹¸
º·
µ´
¶³
s7
¹¸
º· s8
¹¸
º·
-
?valid date in/out
À
?invalid date
¦
¦
¦
¦
¦
¦
¦
¦
¦
¦²
?single
À
?wrong input
-
?double
Y
!Not Available
?
!Price ; c=0
6
!Not Available
,
,
,
,
,
,
µ
!Price ; c=0
-
?Conf ; c<5
À ?CC:type/nbr
-
!valid/notValid
6
!Notice:Conf/Price
Figure 8: TLTS of the composed hotel reservation HR and Credit
card validation Ccv components
5 Conclusion
We have presented a regression testing technique for retest-
ing modified web applications. It selects only necessary test
sequences needed to ensure the correctness of the modified
system. In this work, A two-level abstract model is used
to model the web application and the internal behavior of
the composed components as a Timed Labeled Transition
Systems (TLTS). Modeling with TLTS made it simple for
the regression testing algorithm to select only modification-
related test cases that retest the system. This algorithm is
safe because it selects every test sequence that corresponds
to a different behavior in the modified system. Moreover,
the algorithm updates the test history by eliminating all re-
dundant and non-used test cases; thus, the test history will
remain acceptable in size. Three cases for applying this al-
gorithm are discussed: (1) connecting to a newly established
web service that fulfills a composed component, (2) adding
or removing an operation in any of the composed compo-
nents, (3) modifying the specification of the web applica-
tion. Modifications handled by the algorithm are classified
into three classes: (a) adding an operation, (b) deleting an
operation, (c) fixing a condition or an action.
Further work will implement our technique on a realis-
tic web application to prove its efficiency. We have under-
taken to study some heuristics in order to select pertinent
test sequences and to have a better test coverage .
References
[1] A. Tarhini, H. Fouchal, and N. Mansour. A simple
approach for testing web service based applications.
In 5th international conference on Innovative Internet
Community Systems, I2CS 2005, Paris-France, June
2005.
[2] IBM Web Services Architecture team. Web services
overview. In IBM, 2004.
[3] The Information Security Glossary. Regres-
sion testing. In www.yourwindow.to/information-
security/gl_regressiontesting.htm", November 2005.
[4] H. Leung and L. White. A firewall concept in both
control-flow and data-flow in regression integration
testing. In Proceedings of the Conference on Software
Maintenance, pages 262–271, 1992.
[5] P. Hisa, X. Li, D.C. Kung, C.T. Hsu, Y. Toyoshima
L. Li, and C. Chen. A technique for the selective reval-
idation of oo software. In Journal of Software Mainte-
nance 9, pages 217–233, 1997.
[6] D. Binkley. Semantic guided regression cost reduc-
tion. In IEEE Transactions on Software Engineering
23 (8), pages 498–516, 1997.
[7] F.I. Vokolos and P.G. Frankl. Pythia: a regression test
selection tool based on textual differencing. In Pro-
ceedings of the 3rd International Conference on Reli-
ability, Quality and Safety of Software-Intensive Sys-
tems, pages 3–21, 1997.
[8] R. Gupta, M. J. Harrold, and M.L. Sofa. Program
slicing-based regression testing techniques. In Soft-
ware Testing, Verification and Reliability 6 (2), pages
83–111, 1996.
[9] G. Rothermel, M. J. Harrold, and J. Dehia. Regression
test selection for c++. In software. J. Softw. Testing,
Verif., and Rel. 10(2), June 2000.
[10] N. Mansour and K. El-Fakih. Simulated annealing
and genetic algorithms for optimal regression testing.
In Journal of Software Maintenance, Vol. 11, 19-34.,
June 1999.
[11] N. Mansour and R. Bahsoon. Reduction-based meth-
ods and metrics for selective regression testing. In
Information and Software Technology, Vol 44, No. 7.,
pages 431–443, 2002.
[12] G. Rothermel and M. J. Harrold. A safe, efficient al-
gorithm for regression test selection. In Proceedings
of the conference on software maintenance. CA, pages
358–367, September 1993.
[13] G. Rothermel and M. J. Harrold. A safe, efficient re-
gression test selection technique. In ACM Trans. Soft-
ware Engineering and Methodology, 6(2), pages 173–
210, 1997.
[14] B. Beizer. Testing Techniques. Second Edition. New
York, VanNostrand Reinhold, 1990.
8
... bpws.getVariableData('loanOffer2','payload'…)" (32) <assign name="selectStarLoanOffer"> <copy><from variable="loanOffer2" … /> <to variable="selectedLoanOffer" …/> </copy></assign> <else> (33) <assign name="selectUnitedLoanOffer"> <copy><from variable="loanOffer1" …/> <to variable="selectedLoanOffer" …/> </copy></assign> </else> (31) </if> (29) </scope> (34) <invoke name="replyOutput" partnerLink="client" inputVariable="selectedLoanOffer"…/> (4) </sequence> (2) </process> v1.0 ...
... bpws.getVariableData('loanOffer2','payload'…)" (32) (35) <assign name="selectStarLoanOffer"> <copy><from variable="loanOffer1" …/> <to variable="selectedLoanOffer" …/> </copy></assign> <else> (33) <assign name="selectUnitedLoanOffer"> <copy><from variable="loanOffer1" …/> <to variable="selectedLoanOffer" …/> </copy></assign> </else> (31) </if> (29) </scope> (34) <invoke name="replyOutput" partnerLink="client" inputVariable="selectedLoanOffer"…/> (4) </sequence> (2) </process> v1.1 use (def-use) pairs need to be tested in the all-uses based regression testing of composite service. So another challenge is that, besides identifying the various changes, the various impact they bring to the def-use pairs also needs to be analyzed. ...
... Testing path p1.0 [1] {1, 3,5,6,8,10,11,12,9,7,13,15,17,18,20,22,23,21,24,26,27,25,19,16,14,28,30,32,31,29,34,4, 2} p1.0 [2] {1, 3,5,6,8,10,11,12,9,7,13,15,17,18,20,22,23,21,24,26,27,25,19,16,14,28,30,33,31,29,34,4, 2} p1.1 [1] {1, 3,5,6,8,10,11,12,9,7,13,15,17,18,20,22,23,21,24,26,27,25,19,16,14,28,30,35,31,29,34,4, 2} p1.2 [1] {1, 3,5,6,8,10,11,12,9,7,13,15,17,18,20,22,23,21,24,26,27,25,19,16,14,28,30,32,31,29,34,4, 2} p1.3 [1] {1, 3,5,6,8,10,11,12,9,7,13,15,17,18,20,22,23,21,24,26,27,25,19,16,14,28,30,32,31,29,34,4, 2} p2.0 [1] {1, 3,5,35,37,39,40,41,38,36,6,8,10,11,12,9,7,13,15,17,18,20,22,23,21,24,26,27,25,19,30,32,31,16,14,42,44,46,47,48,49,45,43,50,52,54,53,51,34,4, 2} p2.0 [2] {1, 3,5,35,37,39,40,41,38,36,6,8,10,11,12,9,7,13,15,17,18,20,22,23,21,24,26,27,25,19,30,33,31,16,14,42,44,46,47,48,49,45,43,50,55,57,56, 51, 34, 4, 2} Ref. ...
Article
Full-text available
Composite services evolve for various reasons. Test case selection in the regression testing is an effective technique to ensure the correctness of modified versions meanwhile to reduce the cost of testing. However, few work has studied the test case selection problem based on the data flow testing criteria. In addition, there are three observable kinds of changes during the evolution, including Process change, Binding change and Interface change, which all bring impact to the data flow. To address these issues, a test case selection approach is proposed for regression testing of BPEL (Business Process Execution Language) composite service where all-uses criterion is satisfied and all the three change types are involved. BPEL composite service is modeled with a two-level model in which XCFG (eXtended Control Flow Graph) describes the behavior of BPEL process in the first level and WSDM (Web Service Description Model) depicts the interface information of composite service and partner services in the second level. Change impact analysis is performed to identify the affected definition-use pairs by comparing and analyzing two-level models of the baseline and evolved versions. And testing paths are generated to cover the affected definition-use pairs and select test cases based on the path condition analysis. Empirical result shows that the proposed approach is effective.
... The most commonly used approaches for the selection of regression test cases are either based on manual code analysis or based on expert judgment, but both results in unnecessarily high regression testing cost. A number of regression test selection techniques have been proposed to target different programming paradigms such as procedural programming [4,5,6], object-oriented programming [7,8,9,10], component-based programming [11,12,13] and web applications [14,15,16]. A great number of approaches have also been proposed using different techniques including data flow analysis based [17,18], slicing based [19,20], firewall based [21,22], control flow based [23,24] and differencing based [25,26]. ...
... Therefore regression test suite management is still an open problem where further work could be done in order to improve the existing techniques or propose new techniques that will reduce the effort and cost of the software maintenance activity either by reducing the number of test cases or by selecting a subset of test cases efficiently from an initial test suite. This study is an effort in this direction as all the existing studies [5,9,11,13,15,25,29] try to select a subset of test cases for regression after software undergoes change, but with the proposed study, it is possible to select a subset before software undergoes change and as well as after the change when the software is regression tested. Thus, with the proposed approach, we can run the entire test suite or the selected test cases from each portioned segment or only a single partition or cluster for carrying-out testing activity. ...
... Systematically reviewed the automated RTS techniques for web services [16], and proposed that the ideal techniques should meet the characteristics of composability, end-to-end and automation. In Ruth's work, the RTS techniques based on scenario [31] and protocol [32] were compared, and a RTS technique including initialization stage and key stage based on control flow graphs was proposed, that is, RTS-CFG. In initialization stage, artifacts of the system under testing are collected and control flow graphs are established to represent service invocation logic; in critical stage, test cases are selected according to dangerous edges in control flow graphs. ...
Preprint
Full-text available
Regression testing is required in each iteration of microservice systems. Regression testing selection, which reduces testing costs by selecting a subset from the original test cases, is one of the main techniques to optimize regression testing. Existing techniques mainly rely on the information retrieved from artifacts such as code files and system models. For microservice systems with service autonomy, development method diversity and a large amount of services, such artifacts are too difficultly obtained and costly processed to apply those approaches. This paper presents a regression testing selection approach called MRTS-BP, which needs the API gateway layer logs instead of code files and system models as inputs. By parsing the API gateway layer logs, our approach establishes the service dependency matrix, which in further is transformed into a directed graph with the services as nodes. Then, to find out which test cases are affected by service changes, an algorithm based on belief propagation is presented to compute the quantitative results of service-change propagation from the directed graph. Finally, the relationships between original test cases and service-change propagation results are established to select test cases with three strategies. To evaluate the efficiency of MRTS-BP, the empirical study based on four microservice systems is presented. A typical technique RTS-CFG is compared with MRTS-CFG and four experiments are setup to investigate four research questions. The results show that MRTS-BP can not only reduce the number of test cases by half compared with the retest-all strategy while ensuring the safety, but also save at least 20% testing time costs more than that of RTS-CFG. MRTS-BP is more practical than the techniques relying on the artifacts when the latter cannot be implemented due to the artifacts are difficult to obtain and process.
... Further, service composition issues are unique and error localization can be problematic in such cases. (Tarhini et al 2006(Tarhini et al , 2008) papers discussed a theoretical approach for selection of test cases for web services and applications regression testing. The system is modeled as a timed label transition system or state machine and tried to select unique test cases that can best represent the regression database. ...
Article
Full-text available
Software testing in web services environment faces different challenges in comparison with testing in traditional software environments. Regression testing activities are triggered based on software changes or evolutions. In web services, evolution is not a choice for service clients. They have always to use the current updated version of the software. In addition test execution or invocation is expensive in web services and hence providing algorithms to optimize test case generation and execution is vital. In this environment, we proposed several approach for test cases’ selection in web services’ regression testing. Testing in this new environment should evolve to be included part of the service contract. Service providers should provide data or usage sessions that can help service clients reduce testing expenses through optimizing the selected and executed test cases.
Article
Testing is one of the most important phases in the development of any product or software. Various types of software testing exist that have to be done to meet the need of the software. Regression testing is one of the crucial phases of testing where testing of a program is done for the original test build along with the modifications. In this article, various studies proposed by the authors have been analysed focusing on test cases generation and their approach toward web application. A detailed study was conducted on Regression Test Case Generation and its approaches toward web application. From our detailed study, we have found that very few approaches and methodologies have been found that provide the real tool for test case generation. There is a need of an automated regression testing tool to generate the regression test cases directly based on user requirements. These test cases have to be generated and implemented by the tool so that the reduction in the overall effort and cost can be achieved. From our study, we have also found that regression testing for web applications was not investigated much, but in today's scenario web applications are an integral part of our daily life and so that needs to be tested for regression testing.
Chapter
Web services are a software technology which is based on Service-oriented architecture that is used to provide business functionalities on the web. Thus it is important to ensure proper quality and maintenance of Web Services. Maintenance activity is assumed to be the most expensive activity in software development. Regression testing is a part of maintenance which is done every time whenever a change is made to the software. Regression testing is challenging and time-consuming activity in web services because they are inherently distributed, heterogeneous and dynamic in nature. Thus it is important to reduce regression test effort thereby reducing software maintenance costs. In this paper we have given an efficient approach by which we can effectively carry out regression testing of a web-based application system whenever any changes is made to system.
Chapter
This paper presents an evolutionary method for testing web applications. Although state-based testing has been reported, few papers have addressed modern web applications. In our work, we model web applications by associating features or web pages with states; state transition diagrams are based on events representing state transitions. We formulate the web application testing problem as an optimization problem and use a simulated annealing (SA) metaheuristic algorithm to generate test cases as sequences of events while keeping the test suite size reasonable. SA evolves a solution by minimizing a function that is based on the contradictory objectives of coverage of events, diversity of events covered, and definite continuity of events. Our experimental results show that the proposed simultaneous-operation SA gives better results than an incremental SA version and significantly better than a greedy algorithm.
Conference Paper
Full-text available
The cost of developing and deploying web applications is reduced by dynamically integrating other heterogeneous self-contained web services. However, the malfunctioning of such systems would cause severe losses. This paper presents a technique for building reliable web applications composed of web services. All relevant web services are linked to the component under test at the testing time; thus, the availability of suitable web services is guaranteed at invocation time. In our technique, a web application and its composed components are specified by a two-level abstract model. The web application is represented as Task Precedence Graph (TPG) and the behavior of the composed components is represented as a Timed Labeled Transition System (TLTS). Three sets of test sequences are generated from the WSDL files, the TLTS and the TPG representing the integrated components and the whole web application. Test cases are executed automatically using a test execution algorithm and a test framework is also presented. This framework wraps the test cases with SOAP interfaces and validates the testing results obtained from the web services.
The object-orientated paradigm provides the power for software development but at the same time introduces some brand new problems. One of these problems is that the relationships among classes are more complex and difficult to identify than those in the traditional paradigm. This problem becomes a major obstacle for regression testing of OO software, in which the relationships among classes as well as those between test cases and classes, must be determined a priori. In this paper we propose a new method to select only a fraction of the test cases from the entire test suite to revalidate an OO software system. This method is based on the concepts of class firewall and of marking all the classes ‘touched’ by a test case. From the class firewall, we can identify all of the affected classes after a new version of software is released. Together with the markings, we can also identify all the test cases in the test suite that need to be retested after the software change. A step-by-step process is proposed to identify the relationships between classes and test cases, compute the class firewall, and select only the appropriate test cases for retesting. © 1997 John Wiley & Sons, Ltd.
Article
In corrective maintenance, modified software is regression tested using selected test cases in order to ensure that the modifications have not caused adverse effects. This activity of selective regression testing involves regression test selection, which refers to selecting test cases from the previously run test suite, and test-coverage identification. In this paper, we propose three test-selection methods and two coverage identification metrics. The three methods aim to reduce the number of selected test cases for retesting the modified software. The first method, referred to as modification-based reduction version 1 (MBR1), selects a reduced number of test cases based on the modification made and its effects in the software. The second method, referred to as modification-based reduction version 2 (MBR2) improves MBR1 by further omitting tests that do not cover the modification. The third method, referred to as precise reduction (PR), reduces the number of test cases selected by omitting non-modification-revealing tests from the initial test suite. The two coverage metrics are McCabe-based regression test metrics, which are referred to as the Reachability regression Test selection McCabe-based metric (RTM), and data-flow Slices regression Test McCabe-based metric (STM). These metrics aim to assist the regression tester in monitoring test-coverage adequacy, reveal any shortage or redundancy in the test suite, and assist in identifying, where additional tests may be required for regression testing.
The optimal regression testing problem is one of determining the minimum number of test cases needed for revalidating modified software in the maintenance phase. We present two natural optimization algorithms, namely, a simulated annealing and a genetic algorithm, for solving this problem. The algorithms are based on an integer programming problem formulation and the program's control flow graph. The main advantage of these algorithms, in comparison with exact algorithms, is that they do not suffer from an exponential explosion for realistic program sizes. The experimental results, which include a comparison with previous algorithms, show that the simulated annealing and genetic algorithms find the optimal or near-optimal number of retests within a reasonable time. Copyright © 1999 John Wiley & Sons, Ltd.
Conference Paper
The authors present a methodology for regression testing and function or system testers. The methodology involves regression testing of modules where dependencies due to both control flow and data flow are taken into account. The control-flow dependency is modeled as a call graph and a firewall defined to include all affected modules which must be retested. Global variables are considered as the remaining data-flow dependency to be modeled. An approach to testing and regression testing of these global variables is given, and a firewall concept for the data-flow aspect of software change is defined
Article
Software maintainers are faced with the task of regression testing: retesting a modified program on an often large number of test cases. The cost of regression testing can be reduced if the size of the program is reduced and if old test cases and results can be reused. Two complimentary algorithms for reducing the cost of regression testing are presented. The first produces a program called Differences that captures the semantic change between Certified, a previously tested program, and Modified, a changed version of Certified. It is more efficient to test Differences, because it omits unchanged computations. The program Differences is computed using a combination of program slices. The second algorithm identifies test cases for which Certified and Modified produce the same output and existing test cases that test new components in Modified. The algorithm is based on the notion of common execution patterns. Program components with common execution patterns have the same execution pattern during some call to their procedure. They are computed using a calling context slice. Whereas an interprocedural slice includes the program components necessary to capture all possible executions of a statement, a calling context slice includes only those program components necessary to capture the execution of a statement in a particular calling context. Together with Differences, it is possible to test Modified by running Differences on a smaller number of test cases. This is more efficient than running Modified on a large number of test cases. A prototype implementation has been built to examine and illustrate these algorithms
Article
After changes are made to a previously tested program, a goal of regression testing is to perform retesting based on the modifications while maintaining the same testing coverage as completely retesting the program. We present a novel approach to data flow based regression testing that uses slicing algorithms to explicitly detect definition-use associations that are affected by a program change. An important benefit of our slicing technique is, unlike previous techniques, neither data flow history nor recomputation of data flow for the entire program is required to detect affected definition-use associations. The program changes drive the recomputation of the required partial data flow through slicing. Another advantage is that our technique achieves the same testing coverage with respect to the affected definition-use associations as a complete retest of the program without maintaining a test suite. Thus, the overhead of maintaining and updating a test suite is eliminated. 1. Introduc...
Article
Regression testing is a necessary but costly maintenance activity aimed at demonstrating that code has not been adversely affected by changes. A selective approach to regression testing selects tests for a modified program from an existing test suite. We present a new technique for selective regression testing. Our algorithm constructs control dependence graphs for program versions, and uses these graphs to determine which tests from the existing test suite may exhibit changed behavior on the new version. Unlike most previous techniques for selective retest, our algorithm selects every test from the original test suite that might expose errors in the modified program, and does this without prior knowledge of program modifications. Our algorithm handles all language constructs and program modifications, and is easily automated. 1 Introduction It has been estimated that software maintenance activities account for as much as two-thirds of the overall cost of software production[18]. One...
Article
Regression testing is an expensive but necessary maintenance activity performed on modified software to provide confidence that changes are correct and do not adversely affect other portions of the software. A regression test selection technique chooses, from an existing test set, tests that are deemed necessary to validate modified software. We present a new technique for regression test selection. Our algorithms construct control flow graphs for a procedure or program and its modified version, and use these graphs to select tests that execute changed code from the original test suite. We prove that under certain conditions, the set of tests our technique selects includes every test from the original test suite that can expose faults in the modified procedure or program. Under these conditions our algorithms are safe. Moreover, although our algorithms may select some tests that cannot expose faults, they are at least as precise as other safe regression test selection algorith...