Content uploaded by Filippo Ricca
Author content
All content in this area was uploaded by Filippo Ricca
Content may be subject to copyright.
Content uploaded by Filippo Ricca
Author content
All content in this area was uploaded by Filippo Ricca
Content may be subject to copyright.
Testing Techniques applied to
AJAX Web Applications
Alessandro Marchetto
1
, Paolo Tonella
1
, and Filippo Ricca
2
1
Fondazione Bruno Kessler - IRST, 38050 Povo, Trento, Italy
marchetto|tonella@itc.it
2
Unit`a CINI at DISI
?
, 16146 Genova, Italy
filippo.ricca@disi.unige.it
Abstract. New technologies for the development of Web applications,
such as AJAX, support advanced, asynchronous interactions with the
server, going beyond the submit/wait-for-response paradigm. AJAX im-
proves the responsiveness and usability of a Web application but poses
new challenges to the scientific community: one of them is testing. In
this work, we try to apply existing Web testing techniques (e.g., model
based testing, code coverage testing, session based testing, etc.) to a
small AJAX-based Web application with the purpose of understanding
their real effectiveness. In particular, we try to answer the following ques-
tions: “Is it possible to apply existing testing techniques to AJAX-based
Web applications?”; “Are they adequate to test AJAX applications?”;
and, “What are the problems and limitations they have with AJAX test-
ing?”. Our preliminary analysis suggests that these techniques, especially
those based on white-box approaches, need to be changed or improved
to be effectively used with AJAX-based Web applications.
Keywords: AJAX-based Web applications, Web te sting techniques.
1 Introduction
Traditional Web applications are based on the client-server model: a browser
(client) sends a request asking for a Web page over a network (Internet, via
the protocol HTTP) to a Web server (server), which returns the requested page
as response. During this elaboration time the client must wait for the server
response before visualizing the requested page on the browser, i.e., the model is
based on synchronous communications between client and server.
During the last few years, several Web applications (such as Google Suggest,
Yahoo Instant Search, Google and Yahoo Mail, etc.) have been developed using
a new technology named AJAX [11, 4]. AJAX breaks the traditional Web page
paradigm, in which a Web application can be thought of and modeled as a
graph of Web pages, and invalidates the model of traditional Web applications.
It introduces additional asynchronous server communication to support a more
?
Laboratorio Iniziativa Software FINMECCANICA/ELSAG spa - CINI
responsive user interface: the user interacts directly with items within the page
and the feedback can be immediate and independent of the server’s response.
The validation of Web applications that are based on asynchronous commu-
nication with the server and employ new technologies, such as AJAX, Flash,
ActiveX plug-in components, is an area that deserves further investigation. For
example, until now, little rese arch efforts have focused on how to test Web appli-
cations employing AJAX. Since AJAX does not c omply with the classical Web
application model, several techniques presented in literature will not work any
longer. This s ee ms to be the case of model based Web testing: using a Web
Crawler
3
to extract the model seems not possible anymore. Before devising new
techniques specific of AJAX, we think it is imp ortant to address the following
questions: “Which testing techniques are able to work with AJAX-based Web
applications?” and, “To what degree?”
In this paper, we apply some existing testing techniques to a simple Web
application based on the AJAX technology. Our purp ose is to detect effectiveness
and advantages as well as limitations and problems, of each examined testing
technique.
The rest of the paper is organized as follows. We start, in Section 2, explain-
ing the main characteristics of AJAX. Section 3 presents the existing testing
techniques and tools examined. In Section 4 we analyze each technique in order
to understand and evaluate its applicability to AJAX-applications. In the same
Section we apply the analyzed techniques to a sm all AJAX-application. Section
5 summarizes our preliminary res ults and, finally, in Section 6 we conclude the
paper.
2 AJAX
AJAX (Asynchronous Javascript And XML) is a bundle of existing technolo-
gies used to simplify the implementation of rich and dynamic Web applications.
HTML and CSS are used to present the information, the Document Object
Model is used to dynamically display and interact with the information and the
page structure, the XMLHttpRequest object is exploited to retrieve data from the
Web server, XML is used to wrap data and Javascript is exploited to bind “every-
thing together” and to manage the whole process. With AJAX developers can
implement asynchronous communications between client and server. To achieve
this, client-side scripts and a special AJAX component named XMLHttpRequest
are used. Thanks to AJAX, Web developers can update parts of the client-side
page independently: in AJAX the units of work are the page elements (e.g., text
area, HTML form, DOM structure) rather than the whole page, as happening
with traditional page-based Web applications. For this reason, AJAX breaks the
traditional Web page paradigm. Every element of an HTML page may be tied to
some AJAX action; every action may generate a server request, associated with
3
A Web crawler (also known as a Web spider or robot) is a program that automatically
traverses the Web’s hyperlink structure and retrieves some information for the user.
a URL, so that many HTTP requests can be performed by a single client-side
page.
The main technological novelty of AJAX is the XMLHttpRequest object used
to exchange request and data, wrapped into XML packages, between Web com-
ponents. In particular, through this object a client-side component (e.g., HTML
page) can send HTTP requests to the Web se rver and capture its response in
order to update pieces of the same HTML page. XMLHttpRequest allows to send
asynchronous GET/POST HTTP requests to a given Web server without show-
ing any visible effect to the user and, more importantly, without stoping the
component execution, since a special AJAX engine controls HTTP requests and
responses in background, using an event listener. In other terms, it allows Web
developers to specify event handlers that change the client-side component state
whenever a server response is received asynchronously.
3 Web Testing Techniques
Existing approaches for Web Application testing can be divided into three classes:
white-box, black-box and session-based testing.
3.1 White-Box testing
Similarly to traditional software, white-box testing of Web applications is based
on the knowledge about the internal structure of the system under test. This
approach can be applied to Web applications either by representing the structure
at the high-level, by means of the navigation model (Model-based testing), or
at the low-level, by means of the control flow model (Code-coverage testing).
In the white-box category we consider also Mutation-based testing, which
requires internal knowledge of the Web application under test.
1. Model-based testing [1, 3, 6]. In this approach, reverse engineering tech-
niques and a Web crawler are used to build the Web model of the target
application. The built model is a graph where each node represents a Web
page and each edge represents a link (e.g., HTML links, submits, automatic
redirections). Test cases are extracted by traversing the navigational system
(i.e., the Web model) of the application under test. A test case is composed
of a sequence of pages plus input values.
2. Code-coverage testing [10]. This approach is based on knowledge of the
source code of the Web application under test [10]. In code coverage testing,
the level of coverage reached by a given test suite can be determined by
instrumenting (through trace instructions) the branches in the control flow
model. Since the execution on the server involves one (or more) server side
languages (e.g., Java, PHP, SQL) and the execution on the client involves
additional languages (such as HTML and JavaScript), the control flow model
has different node kinds. Examples of tools used to pe rform code-coverage
testing for Java software are Emma
4
and C lover
5
.
3. Mutation-based testing [2, 8]. Code mutation has bee n used in the lit-
erature for s everal purposes. In the context of Web applications, it can be
used to recover the Web model [2]. In this approach, code mutation is ap-
plied to the server-side code in order to automate the difficult task of model
construction. More traditional uses of mutation consist of applying mutation
operators to the source code in order to generate code mutants. Mutants are
exercised through suites of test cases in order to evaluate their effectiveness
in finding faults. For instance, Elbaum et al.[8] use a fault-based approach
to evaluate the effectiveness of test s uites constructed by means of a session-
based approach. Finally, another use of this defect-injection method is the
selection a subset of test cases, based on a fault-coverage criterion.
3.2 Black-Box testing
The test of the functional requirements can be conducted by considering the Web
application as a black-box. Web applications may have documents describing
the requirements at the user level, such as: use-cases, user stories, functional
requirements in natural language, etc. From these documents, it is possible to
create a list of test cases. Each test case, i.e., ordered list of Web pages plus
user inputs, describes a scenario that can be accomplished by a Web visitor
through a browser. Output pages obtained by navigating the application and
providing the requested inputs are compared with the results expected from
that interaction according to the requirements. In this work, we consider two
black-box approaches: capture/reply and xUnit.
1. Capture and Reply. The most common class of black-box testing tools
provide an infrastructure to support the capture and replay of particular user
scenarios. During black-box testing of Web applications the interaction with
the user c an be simulated by generating the graphical events that trigger
the computation associated with the application interface. One of the main
methods used to obtain this result consists of recording the interactions that
a user has with the Web application and repeating them during the (regres-
sion) testing phase. A lot of functional and regression testing tools, based
on capture/replay facilities, are available as free and commercial software.
Examples are Mercury WinRunner
6
, IBM Rational Robot
7
and MaxQ
8
.
2. xUnit testing. Another approach to black-box testing is based on HttpUnit
9
.
When combined with a framework such as Junit
10
, HttpUnit permits pro-
4
http://emma.sourceforge.net
5
http://www.cenqua.com/clover
6
http://www.merc-int.com
7
http://www-306.ibm.com/software/awdtools/tester/robot
8
http://www.bitmechanic.com
9
http://httpunit.sourceforge.net
10
Http://www.junit.org
grammers to write Java test cases that check the functioning of a Web appli-
cation. HttpUnit is a Java framework well suited for black-box and regression
testing of Web applications, which allows the implementation of automated
test scripts based on assertions. In the xUnit family are also other tools, such
as JsUnit
11
and PHPUnit
12
. Several xUnit tools (e.g., Selenium
13
) use the
capture and reply mechanism to record scripts of test. These scripts can be
completed and enriched with assertions by the user.
3.3 Session-based testing
Another approach to testing Web applications is user-session based testing. It
relies on capturing and replaying real user sessions. This approach avoids the
challenge of building an accurate model of a Web application’s structure. Each
user session is a collection of user requests in the form of URL and name-value
pairs (i.e., input field names and values). A user session begins when a user makes
a new request to a Web application and ends when the user leaves it or the session
times out. To transform a user session into a test case, each logged request is
changed into an HTTP message that can be sent to the Web server. A test case
consists of a set of HTTP requests that are associated with each user session.
Different strategies can be applied to construct test cases from the collecte d
user sessions stored in a log file [5]. The simplest strategy is transforming each
individual user session into a test case. Other, more advanced, strategies are also
possible [7]. An example of access log is given in Figure 5.
4 Testing of AJAX-based Web Applications
In this section we analyze white-box, black-box and session-based testing in
terms of their applicability to AJAX-applications, evaluating limitations and
problems, considering existing tools and applying them to a small “case study”.
Fig. 1. customerID UML model
11
http://www.jsunit.net
12
http://www.phpunit.de
13
http://www.openqa.org/selenium
Fig. 2. customerID screenshot
For our study we selected a small application named customerID
14
. Although
small, it represents a typical AJAX application, which can be analyzed, tested
and described in detail. Figure 1 shows the UML class diagram of this applica-
tion. The custom erID application is composed of:
– a c lient-side HTML page (client.html );
– an HTML form (customer ) used by client.html;
– a Javascript code fragment (JS ) used by client.html;
– a s erver-side page Customer.jsp.
JS retrieves the customer identification number written by the user (see Figure 2)
in the HTML form and uses an XMLHTTPRequest object (activated by the user
through the “submit” button of the form) to send this number to the server-side
page Customer.jsp. Customer.jsp is a JSP component that receives a customer
id from the client and returns first and last name of the customer associated
with the received id, using a XML-based object for data transmission. Then,
the client-side component JS captures this XML package and uses the contained
data to update the client page. In that operation, a fragment of the Javascript
code JS is used to get the data and to update the state of the HTML page
client.html.
Fig. 3. Model-based testing applied to customerID
4.1 Model-based testing
Model-based testing are only partially usable to test AJAX applications, since
the associated model is not adequate for this kind of applications. Existing
14
http://www.ics.uci.edu/∼cs122b/projects/project5/AJAX-JSPExample.html
model-based techniques describe a Web application through a navigation model
composed of Web pages and hyperlinks. The problem is that AJAX applications
are essentially composed of a single page that changes its state according to the
user interactions. Other problems with the model-based approach are: First, ex-
isting Web crawlers, used to build the Web model, are not able to extract the set
of dynamic links created by AJAX applications. Second, client and server com -
ponents tend to exchange small XML data fragments instead of entire HTML
pages. This pec uliarity of AJAX makes it impossible to capture the server re-
sponse in order to extract the hyperlinks of the navigation model, as done in
conventional model-based testing. Third, existing Web models don’t capture the
evolution of a Web page in terms of successive states (i.e. DOM configurations)
and they are not able to represent the XML data exchanged among components.
Summarizing, to perform model-based testing of AJAX-based applications one
needs to:
1. improve the “capability” of the actual Web crawlers;
2. extend the Web model.
These limitations are evident when we apply model-based testing to cus-
tomerID. The model pro duced by the Web crawler is partial (Figure 3 shows
the model extracted according to Ricca and Tonella [6]). It is composed only
of the page client.html, its HTML form and the page customer.jsp. The result
of executing customer.jsp on the server is missing in the model because XML
packages cannot be represented in this model.
4.2 Mutation-based testing
Two studies[2, 9] apply mutation-based testing to Web applications. Bellettini
et al. [2] use mutation to recover the Web mo del, while Sprenkle et al. [9] use
mutation to evaluate the effectiveness of a test suite by inserting artificial defects
(mutations) into the original application. No work to the best of the authors’
knowledge tried to apply mutation to the client-side code of Web applications.
The definition of mutation operators for AJAX could be quite complicated, due
to the possibility of run-time changes of the DOM structure and page content.
We think that mutation-based testing may be useful to support the testing
approaches for AJAX applications. This involves studying and defining AJAX-
specific mutation operators.
We tried to apply the technique proposed by Bellettini et al. [2] to cus-
tomerID, but we found a problem: This testing technique cannot be directly
used with customerID because it applies mutation operators only to server-side
pages and because the response of the server is an XML package and not an
entire page, as expected by the technique.
4.3 Code Coverage testing
In theory, it is possible to apply code coverage testing to AJAX-based applica-
tions. In practice there are some problems. The first is a technological problem.
Fig. 4. Output of our co de-coverage tool applied to client.html
Coverage tools for Web applications need to trace Web code that, often, is a mix
of languages such as HTML, Javascript, JSP and currently, tools with these char-
acteristics are not available. Moreover, currently, it is impossible to use existing
coverage tools to trace dynamic changes of the DOM structure or dynamically
generated code. We think that this list of problems limits in practice the effec-
tiveness of the code coverage testing approach.
Some tools such as Cobertura
15
, JCover
16
, Clover
17
and Emma
18
cannot be
used to test complex and real Web applications bec ause they can trace and
analyze only Java (i.e., server-side) code. An example of code coverage tool for
Javascript is the commercial tool Coverage Validator
19
. It displays statistics for
each Javascript file that is being monitored for code coverage.
A coverage tool able to work with a mix of Web languages is under devel-
opment at our laboratory. Figure 4 shows the output of our tool applied to the
client.html page of the customerID. The code (HTML and Javascript) exercised
during test case execution is traced and shown in light gray. The tool reports
(see top of the figure) som e code-coverage metrics such as condition (60%) and
statement coverage (80%).
When applying our tool to customerID, some problems become evident. It is
difficult to trace dynamic code and dynamic changes of Web pages (e.g., DOM
changes). For instance, we cannot trace the run-time changes of the HTML form
embedded in the client.html page. The reason is that the AJAX component, used
by the same page, dynamically updates the form (name and lastname fields in
Figure 4) through asynchronous communications with the server page. Hence
dynamic changes of the form field values (lines 38-39 on Figure 4) cannot be
traced by the coverage tool.
127.0.0.1 - - [04/Dec/2006:11:31:25 +0100] “GET /ajax1/client.html HTTP/1.1” 200 1916
127.0.0.1 - - [04/Dec/2006:11:31:27 +0100] “GET /ajax/customer.jsp?customerID=134
HTTP/1.1” 200 29
127.0.0.1 - - [04/Dec/2006:11:32:03 +0100] “GET /ajax1/ HTTP/1.1” 200 1563
127.0.0.1 - - [04/Dec/2006:11:32:04 +0100] “GET /ajax1/client.html HTTP/1.1” 200 1917
127.0.0.1 - - [04/Dec/2006:11:32:07 +0100] “GET /ajax1/customer.jsp?customerID=245
HTTP/1.1” 200 29
Fig. 5. Tomcat log-file of customerID
4.4 Session-based testing
This testing approach is apparently applicable to AJAX applications, because in
a log-file we can capture both kinds of HTTP requests: traditional (triggered by
15
http://cobertura.sourceforge.net
16
http://www.mmsindia.com/JCover.html
17
http://www.cenqua.com/clover
18
http://emma.sourceforge.net
19
http://www.softwareverify.com/javascript/coverage/feature.html
user clicks), as well as AJAX-sp e cific (triggered by AJAX-components). How-
ever, session-based testing techniques are adequate to verify only non-interacting,
synchronous request-response pairs, because of two kinds of limitations:
1. Using (only) log-files information it is not possible to reconstruct the state
of Web pages that are modified during the execution of a given application.
The reason is that the data exchanged between client and server in a given
AJAX application are only “pieces of data” wrapped into XML messages,
that are eventually turned into page changes;
2. Some techniques used to mix log file information (i.e., sequence of hyperlinks
and input values) cannot be used to generate new navigation sess ions (i.e.,
new sequences of links and input values) useful to e xercise the application
under test. In fact, it may be hard to reproduce the context where the log
information used in the original application can be re-inserted in a different
scenario.
Figure 5 shows a fragment of the Tomcat log-file for the customerID ap-
plication. Given this log file, we can reply the two customerID usage sessions
performed by the user but we cannot take advantage of the information in the
XMLHTTPRequest object used by customerID. For this reason, we can not re-
peat its behavior. Moreover, we can use the log information to verify each single
response to the HTTP requests stored in Tomcat log files. But, we cannot use
it to tes t the entire application customerID, since it is impossible to reconstruct
the state of the application when each HTTP request was issued and when the
response was received. Furthermore, by mixing the log-extracted HTTP requests
we might end up with inconsistent requests, since knowledge of the AJAX-objects
used by customerID cannot be derived from log-files only. Some examples of test
cases derived from the log-file in Figure 5 are the following:
1. GET client.html → verify the HTML code s ent by the server on response;
2. GET client.html → send ID=134 to customer.jsp using a GET request →
verify the returned XML package;
3. GET client.html → verify the HTML code s ent by the server on response;
4. GET client.html → send ID=134 to customer.jsp using a GET request →
send ID=234 to customer.jsp using a GET request → verify the returned
XML package.
4.5 Capture and Reply
Capture and replay is in principle applicable to AJAX-based Web applications,
since it exercises the application from a user point-of-view using the GUI. How-
ever, the real applicability to AJAX-based Web applications depends on the
actual testing tool in use. Several implementations of this technique would need
to be improved to be eff ectively used to test AJAX applications. To be success -
fully applied to AJAX applications, a capture and reply tool should:
1. support Javascript;
function testupdateFirstLastName() {
<?xml version=‘‘1.0’’ encoding=‘‘UTF-8’’?>
<LogiTest:test
xmlns:LogiTest=‘‘http://www.logitest.org’’>
<LogiTest:name>Untitled</LogiTest:name>
<LogiTest:description />
<LogiTest:resetCookies>false</LogiTest:resetCookies>
<LogiTest:resource
url=‘‘http://localhost:8080/ajax1/client.html’’
method=‘‘GET’’ delay=‘‘0’’ />
</LogiTest:test>
Fig. 6. LogiTest applied to customerID
2. be able to capture dynamic events associated with user input;
3. be able to capture dynamic changes of the DOM structures;
4. be able to perform asynchronous HTTP requests to the Web server.
Fig. 7. Selenium applied to customerID
Examples of capture and reply tools are LogiTest
20
, Maxq
21
and Badboy
22
.
These tools don’t work well with AJAX applications. Logitest doesn’t sup-
port Javascript while Maxq cannot record dynamic events. Badboy supports
Javascript but it is not able to capture some dynamic events (e.g., “onblur”
on form input fields) and run-time changes of the DOM structures. Thus, it is
not adequate to test AJAX applications. The commercial tool eValid
23
and the
tool Origsoft
24
promise to test the AJAX applications. Indeed, they are able to
capture dynamic e vents and store dynamic DOM changes.
20
http://logitest.sourceforge.net
21
http://maxq.tigris.org
22
http://www.badboy.com.au
23
http://www.soft.com
24
http://www.origsoft.com
We have tried to apply a tool of this category to customerID. Figure 6 shows
the navigation-script stored by LogiTest for an usage session of our customerID.
It is clear that this script is not adequate to test customerID since it doesn’t
contain information related to the asynchronous HTTP requests performed by
the application.
4.6 xUnit testing
The xUnit approach is also in principle applicable to test AJAX applications.
Actually, it focuses on the functional behavior rather than the implementation.
However, the real applicability to AJAX Web applications depends on the actual
implementation of the tool in use. To be used with AJAX-applications, xUnit
tools must support, at least, Javascript, asynchronous HTTP requests and DOM
inspection.
Examples of xUnit testing tools are: Latka, HTTPUnit, InforMatrix, HTM-
LUnit, JsUnit, Canoo WebTest, squishWeb and Selenium. Latka
25
and HTTPUnit
26
cannot be used to test AJAX-applications because they are not able to manage
asynchronous HTTP requests and DOM inspection. Some tools such as Infor-
Matrix
27
, HTMLUnit
28
and Canoo WebTest
29
have be en recently improved to
support Javascript and AJAX components. Unfortunately, their Javascript sup-
port is still limited and in real AJAX applications they can hardly be applied.
Instead, software such as squishWeb
30
and Selenium
31
can be used to test AJAX
applications, since they fully support Javascript, asynchronous HTTP requests
and DOM inspection.
We have tried to apply some tools of this category to customerID. Infor-
Matrix and HtmlUnit can not be used to test customerID: they don’t support
some DOM-events actions, while they partially support Javascript. Differently,
Selenium, thanks to the assertion “wait conditions”, can be success fully used to
test customerID. Figure 7 shows two screenshots of Selenium. The considered
test case is the following:
1. load client.html → type the number “123” in the form ID → click the submit
button (i.e., send data to the customer.jsp) → verify the form name has been
updated with “John”.
In this example, we use two different assertions to verify the output of the
above test case. In the first case , we use a conventional “assert” command instead
of the correct “waitForValue” (see Figure 7, right). Without this “waitForValue”
25
http://jakarta.apache.org/commons/latka
26
httpunit.sourceforge.net
27
http://www.informatrix.ch
28
http://htmlunit.sourceforge.net
29
http://webtest.canoo.com
30
http://www.froglogic.com
31
http://www.openqa.org/selenium-ide/
the test case verification fails (see Figure 7, left) since the AJAX application cus-
tomerID uses asynchronous communications and thereby a conventional “assert”
cannot be used to capture the server response.
5 Discussion
Testing adequate problems tools customerID
Model-based no Web models extracted research not ok
are partial; existing Web
crawlers are not able
to download site pages
Mutation-based no mutant operators are not-existing not ok
never being applied
to client Web code;
the application of
mutant operators
is difficult
Code Coverage partially it is difficult to cover Javascript: Coverage partially ok
dynamic events and validator
DOM changes; coverage Java: Cobertura,
tools managing a mix Emma, Clover, etc.
of languages are not Languages mix :
available not available
Session-based no it is impossible to research not ok
reconstruct the state
of the Web pages
using only log-files
Capture&Reply yes Javscript, asynchronous not ok : Maxq, it dep end s
and xUnit HTTP requests and HTTPUnit, on the
DOM analysis are InforMatrix etc. tool
not always partially ok : Badboy, implementation
supported HTMLUnit etc.
ok: squishWeb,
Selenium etc.
Table 1. Web testing techniques applied to AJAX-based applications
Table 1 summarizes our preliminary analysis of e xisting testing techniques
applied to AJAX Web applications:
1. model-based: test cases are derived from the Web model. The model doe sn’t
consider all the states that a single HTML page may reach during the exe-
cution of the Web application. So, this technique appears applicable but not
adequate to test AJAX-applications.
2. mutation-based: to our knowledge, no work defines mutation operators
that apply to client-side code of Web applications. Moreover the application
of mutation operators is complicated by the specific nature of the AJAX Web
applications (HTML pages can dynamically change their DOM structure and
content). Additionally, specific tools for Web applications are not available.
So, this technique is promising, but far from being available in practice.
3. coverage-based: this technique is applicable and adequate to test AJAX
applications. However, its real effectiveness has to be verified in practice since
it is difficult to instrument and trace AJAX dynamic code (in particular,
dynamic e vents and DOM changes). Another problem is that coverage tools
for Web applications must trace a mix of languages and currently, tools
with these characteristics are not available. For this reason, this technique
is considered only partially adequate to test this kind of software.
4. session-based: in this technique test cases are derived using the data recorded
in the log-files. Since in a log-file we can capture traditional HTTP requests
as well as AJAX-specific ones, this technique is apparently applicable to test
AJAX applications. However, session-based testing is not fully adequate be-
cause it is hard to reconstruct the state of the Web pages that are exercised
during the exec ution of a given application using only log-files information.
5. capture/reply and xUnit: this category of tools is adequate to test AJAX
applications because it does not consider the internal structure of the tar-
get application. The capture/reply and xUnit tools verify the functionalities
of the Web applications based only on requirements and expected output
(black-box testing). However, often, existing tools don’t support AJAX or
the support is still limited. A tool of this category, to be used with AJAX-
based Web applications should support at least: Javascript, asynchronous
HTTP requests and DOM inspection.
6 Conclusions
In this paper we have applied some existing testing techniques to a simple AJAX
We b application with the purpose of detecting effectiveness and advantages, as
well as limitations and problems, of each examined technique.
The results of our preliminary analysis can be summarized as follows: model
and session-based testing techniques need to be modified and improved to let
them test AJAX Web applications. Mutation-based testing needs to be adapted
to be used with the client-side components of AJAX applications. Code-coverage
testing can be only “partially” used with AJAX-applications: (1) the dynamism
of this kind of technology limits its effectiveness; (2) tools are not available to
manage mix of languages as required. Currently, capture/reply and xUnit testing
are the only tools able to work with AJAX. However, some of them have to be
extended/improved to support Javascript, dynamic changes/events of the DOM
and AJAX-c omponents.
Our preliminary analysis suggests that to test AJAX-applications new ap-
proaches/tools are needed, since the existing ones have severe limitations and
problems, especially white-box and session-based testing techniques.
References
1. A. Andrews, J. Offutt, and R. Alexander. Testing Web Applications by Modeling
with FSMs. Software and System Modeling, Vol 4, n. 3, July 2005.
2. C. Bellettini, A. Marchetto, and A. Trentini. Dynamic Extraction of Web Appli-
cations Mo de ls v ia Mutation Analysis. Journal of Information, 2005.
3. C. Bellettini, A. Marchetto, and A. Trentini. TestUml: User-Metrics Driver Web
Applications Testing. 20th ACM Symposium on Applied Computing (SAC 2005),
Santa Fe, New Mexico, USA. March 2005.
4. J. Eichorn. Understanding AJAX: Using JavaScript to Create Rich Internet Ap-
plications. Prentice Hall, 2006.
5. S. Elbaum, G. Rothermel, S. Karre, and M. Fisher. Leveraging user session data to
support Web Applications Testing. IEEE Transactions on Software Engineering,
Vol. 31, n. 3, March 2005.
6. F. Ricca and P. Tonella. Building a Tool for the Analysis and Testing of Web
Applications: Problems and Solutions. Tools and Algorithms for the Construction
and Analysis of Systems (TACAS’2001), Genova, Italy. April 2001.
7. E. Sampath, S. Gibson, S. Sprenkle, and L. Pollock. Coverage Criteria for Testing
Web Applications. Technical Report 2005-17, Computer and Information Sciences,
University of Delaware., 2005.
8. S. Sprenkle, E. Gibson, S. Sampath, and L. Pollock. Automated Replay and Failure
Detection for Web Applications. 20th IEEE/ACM International Conference on
Automated Software Engineering (ASE 2005), Usa. 2005.
9. S. Sprenkle, E. Gibson, S. Sampath, and L. Pollock. A Case Study of Automatically
Creating Test Suite from Web Applications Field Data. Workshop on Testing,
Analysis and Verification of Web Services and Applications (TAV-WE B 2006),
Usa. 2006.
10. P. Tonella and F. Ricca. A 2-Layer Model for the White-Box Testing of Web
Applications. International Workshop on Web Site Evolution (WSE’04), Illinois,
USA. September 2004.
11. E. Woychowsky. AJAX: Creating Web Pages with Asynchronous JavaScript and
XML. Bruce Perens’ Open Source Series, 2006.
