ArticlePDF Available

A Survey of Service Oriented Architecture Systems Testing

Authors:

Abstract and Figures

Service oriented architecture (SOA) is one of the latest software architectures. This architecture is created in direction of the business requirements and removed the gap between softwares and businesses. The software testing is the rising cost of activities in development software. SOA has different specifications and features proportion of the other software architectures. First this paper reviews SOA testing challenges and existing solution(s) for those challenges. Then that reports a survey of recent research to SOA systems testing, that covers both functional and non-functional testing. Those are presented for different levels of functional testing, including unit, integration, and regression testing.
Content may be subject to copyright.
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
DOI : 10.5121/ijsea.2012.3602 19
A SURVEY OF SERVICE ORIENTED ARCHITECTURE
SYSTEMS TESTING
Ebrahim Shamsoddin-Motlagh
1
1
Computer Engineering Department, Faculty of Engineering, Science & Research
Branch, Islamic Azad University, Tehran, Iran
e.shamsoddin@srbiau.ac.ir
ABSTRACT
Service oriented architecture (SOA) is one of the latest software architectures. This architecture is created
in direction of the business requirements and removed the gap between softwares and businesses. The
software testing is the rising cost of activities in development software. SOA has different specifications and
features proportion of the other software architectures. First this paper reviews SOA testing challenges and
existing solution(s) for those challenges. Then that reports a survey of recent research to SOA systems
testing, that covers both functional and non-functional testing. Those are presented for different levels of
functional testing, including unit, integration, and regression testing.
KEYWORDS
Service Oriented Architecture Systems Testing, SOA
1. INTRODUCTION
Arasanjani, Borges and Holley define SOA as follows [1]: “SOA is the architectural style that
supports loosely coupled services to enable business flexibility in an interoperable, technology-
agnostic manner. SOA consists of a composite set of business-aligned services that support a
flexible and dynamically re-configurable end-to-end business processes realization using
interface-based service descriptions.”
System services have features in the design and implementation, these features include: Service
reusability, Standardized service contract, Service loose coupling, Service abstraction, Service
composition, Service autonomy, Service statelessness and service discoverability capabilities.
The purpose of this paper is to find the testing challenges in the SOA systems and existing
solution(s) and to review recent research in the test of SOA systems. The paper is structured as
follows. Section 2 challenges are expressed in the SOA Testing, and then the solutions to
eliminate or minimize problems are expressed in these challenges. Section 3 a review related
work of SOA Testing. Finally, Section 4 summarizes the paper and outline suggests future
research steps.
2. SOA SYSTEMS TESTING CHALLENGES
The SOA system has different nature and the specific characteristics than the traditional system of
the test system; it's harder and needs more time. The test facilitates and abilities at the SOA
system testing should be recognized and solution(s) should be presented for testing challenges.
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
20
The key issues of testability limits of the SOA systems include: dynamicity and adaptiveness,
lack of observability of the service code and structure, lack of control, lack of trust, new aspects
of testing, test cost, different stakeholders [2, 3, 4, 5, 6, 7].
2.1. Functional Testing Challenges
The challenges of functional testing can be reported: All inputs and output may be very difficult
to the system testing, Asynchronous in the SOA systems, rapid growth and large system (the
development and management should be understood to be the test team) and need the tools and
knowledge to work the system workflow [8].
The services don't have the service interface in unit testing [8], which this makes quality
assurance team will have to be the implemented skills with to be able to produce good test, test
objects, and the required test data. Another the service unit testing differences in input\output
types with component testing and that complex [6], more test data generation techniques work on
simple data, but in reality in complex systems such as SOA systems have complex inputs such as
XML also used to inputs.
2.2. Non-Functional Testing Challenges
Non-functional properties of the system such as: availability, performance, applicability,
maintenance capabilities, and portability. SOA has problems such as non-functional testing is
impossible or difficult to determine a service workload parameters at service level agreement
(SLA), the existing problems in the network and impact on system performance, system security
(due to the decentralized system and the system is in the distributed system with different
frameworks) [8]. The challenges of the test reliability can be reported [9]:
1- It may need a lot of time to test in the real execution as the faults may not occur in many
situations.
2- Web services may involve many outside service providers who charge their service
provided and it may a lot of cost.
3- The services in BPEL from different organizations should cooperate to achieve business
goals. Their execution should be transactional.
4- Traditional faults generation techniques can only generate low level error regardless the
business process, so that it makes difficult to test the whole process because the faults
generated may not affect the business process intentionally.
2.3. Existing Solutions
The solutions of the functional testing challenges can be reported:
Among the proposed solutions for problems and challenges in the research include the functional
testing procedures are updated at SOA Systems, and the existing methods are automated. The
tools is used to performing complex actions and the integrity of system is able to management are
produced. The monitoring system operated at all levels, and used the ESB capabilities for
functional testing system. If needed and usability of other existing technologies (such as JMS
middleware) was used.
Another solution is to improve the system new implementations [10]. Futures of the Internet need
to make SOA testbed for large systems and reduce their cost of test. The testbed be required to
validate and the integrity of the future Internet. The network should be loosely coupled, networks
will be heterogeneous, and there will have capabilities in the program layer, and complexity will
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
21
be greater in the composition. The systems need to automatic composition of services and
management processes, there need to combine the testbed capabilities.
At the beginning of creating a production SOA system, the test team haven't high collaboration,
but must be present to understand the goals and business processes, in order to properly and
efficiently carry out the test, and that prevent the additional costs to the system development
process.
2.4. Summarized
Table 1 shows the SOA Testing challenges with testing levels in the SOA systems.
Table 1. SOA Testing challenges with testing levels of the SOA systems.
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
22
3. SOA SYSTEMS TESTING
SOA system testing should be performed of aspects functional testing and non-functional testing,
the functional testing has different levels, the levels include unit testing in individual services and
the combined services, integration testing and regression testing.
This part of paper is checked SOA system testing of the functional aspects (in levels) and non-
functional aspects, and then the test automation tools is described in the SOA system. Then
automatic test data generation techniques are reviewed for traditional software. Finally, these
results are shown in tables.
3.1 Unit Testing
Numbers of investigations in the unit testing have been active for test automation, in their
attempted to automate process or processes of testing. The researches [11, 12, 13, 14, 15, 16]
performed unit testing on WSDL file.
3.2 Integration Testing
Numbers of existing researches [17, 18, 19, 20, 21, 22, 23] performed BPEL-based testing in the
system with operations graph.
In studies [24 and 25] implemented test at combining web services used to high level Petri net
and specifications BPEL. You can generate test cases from web service automata (WSA)
automatically [26]; WSA can be used to define the operational logic in BPEL.
Numbers of test frameworks have been prepared for SOA testing, than these are performed the
SOA testing with the best way [27, 28].
The DFTT4CWS tools automatically find unusual data flow [29] and the test paths is generated
data flow testing with all of cover criteria types. In reference [30], BPEL file is mapped DOM
object tree to the EMF activities tree. The WebMov is set of tools modelling, evaluates and tests
web services composition [31]. One paper was expressed computational strategy for the
generation complete computational paths of BPEL based on Tabu Search and Genetic Algorithms
to generate test data [32].
The research [33] provided an approach to design test cases based on functional properties of
high-level business process model. The study [34] proposed an approach for reducing the costs to
test such applications, and how can semantic stubs enable the client test suite to be partitioned
into subsets, some of which don’t use to execute remote services. Model driven approach is
presented in [35], this approach to generate executable test cases from the given express business
processes.
3.3 Regression Testing
Researches [36, 37] are proposed an approach to determine the changes use to extensible BPEL
flow graph (XBFG) of control flow and to compare the paths in a new version of service
composition with the old version.
Testing rules and monitoring rules include: checking the functional characteristics services,
checking quality of service (QoS), checking interoperability services and service evolution [38].
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
23
3.4 Non-Functional Testing
Non-functional requirements include [39]: the needed data to fulfill the monitoring goal is
intercepted. Monitoring mechanisms attempt the performance isn’t influenced of unmonitored
elements and performance is influenced of the monitored elements remains to be minimal. The
changes responses are in the monitoring goal and environment topology. Instrumentation must be
transparent and performed on demand.
System security is one of the characteristics non-functional SOA systems. The paper [40]
presented a preliminary approach towards an evaluation framework for SOA security testing
tools.
A research proposes a technique on how do reliability test define of composite service in BPEL
from the view of business semantics with little cost using fault injection [9]. This paper focus on
how the reliability problems find relate with business process, the called semantics as the
problems are not pure coding error but faults related to business process. In addition, the
behaviour of composite services in BPEL is analyzed when there are faults in the orchestrated
services invoked.
3.5 Automatic testing tools
Numbers of existing produced tools was created to test SOA systems automatically. For example
TASSA is a framework for automatic testing in functional and non-functional specifications of
service-based applications [41]. It provides end-to-end testing of Service layer, Service
Composition and coordination and business process. Another tool is WSOTF presented for the
automated testing [42]. WSOTF is an automatic conformance testing tool with timing constraints
from a formal specification of web services composition that is implemented by an online testing
algorithm.
In the study [43] is expressed test approach described in BPEL web service composition. The
paper [44] is proposed to generate a testbed for service-oriented systems that takes into account a
mobility model of nodes in the network which the accessed services are deployed.
The study [45] is a framework and its supporting tool for automatically generating and executing
web-service requests and analyzing the subsequent request-response pairs.
The study [46] is proposed an approach to combine the accessibility technologies in graphical
applications (GAPs) for a visualization mechanism enables nonprogrammers to generate unit test
cases in web services by drag-and-drop operations on graphical user interface (GUI). In the
reference [47] is testing techniques to generate a set of test cases for web services automatically.
The techniques presented here explore data perturbation of Web services messages upon data
types, integrity and consistency.
3.6 Summarized
In the paper [48] is expressed a survey to explore cloud services testing methods. The paper [49]
is expressed a review to identify SOA testing researches with dynamic binding, that paper
performed manually and automatically search in journals, conferences and etc.
Methods described in Table 2 with different levels of test coverage, SOA system testing and see
in them. In A service may be provided in the composition of services using BPEL file, the reason
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
24
test on the some parts of this table is on the integration testing (the BPEL file) also put on the unit
testing capabilities.
Table 2. SOA systems testing Methods at levels testing
Method
Level Testing
Unit
Testing
Integration
Testing
Regression
Testing
Non-
Functional
Testing
DOM tree [11, 14],
DbC [13, 16], WS-
TAXI [15], GAP [46],
GenAutoWS [47],
[45]
Generate
test cases
by WSDL
-
-
-
SAT Solver [21]
-
Generate test cases
by processes and
user activity
Use to save
test cases
-
CPM [33], [20, 27, 28,
29, 34, 35]
-
Generate test cases
for processes
business
-
-
SXM [22, 23], BTA
[25], TestGen-IF [43],
WSA [26], [17, 18,
19, 24, 30], Tabu [32]
BPEL
testing
Generate test cases
for BPEL service
-
-
TASSA [41]
Layer
service
testing
Generate test cases
at orchestration
and BPEL service
-
Layers testing,
Coordination
and service
composition
WSOTF [42]
Analysis
WSDL
Generate test cases
at specification
system
-
-
XBFG [36,37]
-
-
Select test
cases
-
[40]
-
-
-
Security
[9]
-
-
-
Reliability
[44]
-
-
-
Generate
specification in
mobile system
model
4. CONCLUSIONS
The part of tow this paper was expressed challenges and limitations of the SOA systems testing,
and the existing solutions to solve some of them. The SOA systems testing challenges include:
dynamicity and adaptiveness, lack of observability of the service code and structure, lack of
control, lack of trust, new aspects of testing, test cost, different stakeholders. The part of three this
paper was expressed SOA systems testing at two aspects of functional testing and non-functional
testing. The functional testing was investigated in three different parts of the unit, integration and
regression. In the last of chapters the results of these researches showed in the relevant tables.
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
25
Manually test cases generation and manually test operations are a difficult and time consuming
and the dynamic nature of the SOA systems cause the test cases is generated after some time lose
their usability. To resolve this problem need to create a dynamic and automatic way to generate
test cases in orchestration of SOA. Future works will propose specific approaches for specific
software in the software logic or improve existing approaches for specific software. Another
create test cases generation framework for SOA systems. Future work can be integration of
available tools.
ACKNOWLEDGEMENTS
The author would like to thank specially Dr. Seyed Hasan Mirian Hossienabadi who has extended
his support for successful completion of this paper.
REFERENCES
[1] Torry Harris Business Solutions (THBS) Company (2007). SOA Test Methodology [White paper].
Retrieved from www.thbs.com/pdfs/SOA_Test_Methodology.pdf
[2] Hewlett-Packard (HP) Development Company (2007, March). Testing service oriented architecture
applications and services [White paper]. Retrieved from
http://i.i.com.com/cnwk.1d/html/itp/HP_4AA1-1389ENW_Testing_SOA_WP_5966.pdf
[3] IBM Corporation Software Group (2007, May). Testing SOA applications with IBM Rational quality
management. White paper. Retrieved from
ftp://public.dhe.ibm.com/software/rational/web/whitepapers/10708268_Testing_SOA_WP_acc.pdf
[4] Greiler, M., Gross, H. G., Nasr, K. A. (2009). Runtime Integration and Testing for Highly Dynamic
Service Oriented ICT Solutions An Industry Challenges Report. 2009 Testing: Academic and
Industrial Conference - Practice and Research Techniques, 51-55. Doi: 10.1109/TAICPART.2009.16
[5] Bertolino, A., & Polini, A. (2009). SOA Test Governance: enabling service integration testing across
organization and technology borders. IEEE International Conference on Software Testing Verification
and Validation Workshops, 277-286. Doi: 10.1109/ICSTW.2009.39
[6] Canfora, G., & Penta, M. D. (2008). Service Oriented Architectures Testing A Survey. This work is
partially founded by the European Commission VI Framework IP Project SeCSE (Service Centric
SystemEngineering) (http://secse.eng.it), Contract No. 511680, and by the Italian Department of
University and Research (MIUR) FIRB Project ARTDECO. Retrieved from
http://rcost.unisannio.it/rcost_www/mdipenta/papers/lncs09.pdf
[7] Canfora, G., & Penta, M. D. (2006, March|Appril). Testing Services and Service-Centric Systems:
Challenges and Opportunities. IEEE Computer Society, 10-17. Doi: 10.1109/MITP.2006.51.
[8] Ribarov, L., Manova, I., Ilieva, S. (2007, September 21-23). TESTING IN A SERVICE-ORIENTED
WORLD. The International Conference on Information Technologies (InfoTech-2007), Bulgaria vol.
1, 1-10. Retrieved from
http://dspace.ou.nl/bitstream/1820/1067/1/TestingInAServiceOrientedWorld.pdf
[9] Wang, Y., Ishikawa, F., Honiden, S. (2010). Business Semantics Centric Reliability Testing for Web
Services in BPEL. IEEE 6th World Congress on Services, 237-244. Doi:
10.1109/SERVICES.2010.88
[10] Magedanz, T., Schreiner, F., Wahle, S. (2009). Service-Oriented Testbed Infrastructures and Cross-
Domain Federation for Future Internet Research. 2009 IFIP/IEEE Intl. Symposium on Integrated
Network Management Workshops, 101-106. Doi: 10.1109/INMW.2009.5195944
[11] Bai, X., Dong, W., Tsai, W. T., Chen, Y. (2005). WSDL-Based Automatic Test Case Generation for
Web Services Testing. 2005 IEEE International Workshop on Service-Oriented System Engineering
(SOSE’05), 1-6. Doi: 10.1109/SOSE.2005.43
[12] Jiang, Y., Li, Y. N., Hou, S. S., Zhang, L. (2009). Test-Data Generation for Web Services Based on
Contract Mutation. 2009 Third IEEE International Conference on Secure Software Integration and
Reliability Improvement SSIRI 2009 Short Paper, 281-286. Doi: 10.1109/SSIRI.2009.49
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
26
[13] Ma, C., Du, C., Zhang, T., Hu, F., Cai, X. (2008). WSDL-Based Automated Test Data Generation for
Web Service. International Conference on Computer Science and Software Engineering, 731-737.
Doi: 10.1109/CSSE.2008.790
[14] Dong, W. (2009). Testing WSDL_based Web Service Automatically. World Congress on Software
Engineering, 521-525. Doi: 10.1109/WCSE.2009.133
[15] Bartolini, C., Bertolino, A., Marchetti, E. (2009). WS-TAXI: a WSDL-based testing tool for Web
Services. International Conference on Software Testing Verification and Validation, 326-335. Doi:
10.1109/ICST.2009.28
[16] Noikajana, S., & Suwannasart, T. (2009). An Improved Test Case Generation Method for Web
Service Testing from WSDL-S and OCL with Pair-wise Testing Technique. 33rd Annual IEEE
International Computer Software and Applications Conference, 115-123. Doi:
10.1109/COMPSAC.2009.25
[17] Mei, L., Chan, W.K., Tse, T.H. (2008, May 10-18). Data Flow Testing of Service-Oriented Workflow
Applications, ICSE ’08, Leipzig, Germany, 371-380. Doi: 10.1145/1368088.1368139
[18] Lertphumpanya, T., & Senivongse, T. (2008, May). Basis Path Test Suite and Testing Process for
WS-BPEL. WSEAS TRANSACTIONS on COMPUTERS. ISSN: 1109-2750, Issue 5, Volume 7,
483-496. Retrieved from www.wseas.us/e-library/transactions/computers/2008/26-156.pdf
[19] Yuan, Y., Li, Z., Sun, W. (2006). A Graph-search Based Approach to BPEL4WS Test Generation.
Proceedings of the International Conference on Software Engineering Advances (ICSEA'06). Doi:
10.1109/ICSEA.2006.261270
[20] Bartolini, C., Bertolino, A., Elbaum, S., Marchetti, E. (2011). Bringing white-box testing to Service
Oriented Architectures through a Service Oriented Approach. The Journal of Systems and Software,
84, 655668. Doi:10.1016/j.jss.2010.10.024
[21] Radhakrishnan, K., & Podorozhny, R. (2009, February 16). Automatic test case generation for web
service processes using a SAT solver (Report Number TXSTATE-CS-TR-2009-13).
https://digital.library.txstate.edu/bitstream/handle/10877/2581/fulltext.pdf
[22] Ma, C., Wu, J., Zhang, T., Zhang, Y., Cai, X. (2008) Automatic Test Case Generation for BPEL
Using Stream X-Machine. International Journal of u- and e- Service, Science and Technology, 27-36.
Retrieved from http://www.sersc.org/journals/IJUNESST/vol1_no1/papers/04.pdf
[23] Ma, C., Wu, J., Zhang, T., Zhang, Y., Cai, X. (2008). Testing BPEL with Stream X-machine.
International Symposium on Information Science and Engieering, 578-582. Doi:
10.1109/ISISE.2008.201
[24] Dong, W.L., YU, H., Zhang, Y.B. (2006). Testing BPEL-based Web Service Composition Using
High-level Petri Nets. Proceedings of the 10th IEEE International Enterprise Distributed Object
Computing Conference (EDOC'06), 441 444. Doi: 10.1109/EDOC.2006.59
[25] Dong, W. (2009). Test Case Generation Method for BPEL-based Testing. 2009 International
Conference on Computational Intelligence and Natural Computing, 467-470. Doi:
10.1109/CINC.2009.229
[26] Zheng, Y., Zhou, J., Krause, P. (2007, September). An Automatic Test Case Generation Framework
for Web Services. JOURNAL OF SOFTWARE, VOL. 2, NO. 3, 64-77. Retrieved from
http://epubs.surrey.ac.uk/1975/1/fulltext.pdf
[27] Lee, Y. (2009). 2-Layered SOA Test Framework Based on BPA1-Simulated Event. Fifth International
Joint Conference on INC, IMS and IDC, 1058-1063. Doi: 10.1109/NCM.2009.337
[28] Mei, L. (2009, May 16-24). A Context-Aware Orchestrating and Choreographic Test Framework for
Service-Oriented Applications. ICSE’09, Vancouver, Canada, 371-374. Doi: 10.1109/ICSE-
COMPANION.2009.5071024
[29] Hou, J., & Xu, L. (2009). DFTT4CWS: A Testing Tool for Composite Web Services Based on Data-
Flow. Sixth Web Information Systems and Applications Conference, 62-67. Doi:
10.1109/WISA.2009.19
[30] Huang, J, & Gong, Y. (2010). An EMF Activity Tree Based BPEL Defect Pattern Testing Method.
2nd International Conference on Computer Engineering and Technology, 7, 468-471. Doi:
10.1109/ICCET.2010.5485536
[31] Cavalli, A., Cao, T.D., Mallouli, W., Martins, E., Sadovykh, A., Salva, S., Za¨ıdi, F. (2010).WebMov
A dedicated framework for the modelling and testing of Web Services composition. IEEE
International Conference on Web Services, 377-384. Doi: 10.1109/ICWS.2010.24
[32] Bo, Y., Ye-mei, Q., Ge, Y., Chang, G. (2009). Tabu Search and Genetic Algorithm to Generate Test
Data for BPEL Program. Computational Intelligence and Software Engineering (CiSE), 1-6. Doi:
10.1109/CISE.2009.5363674
International Journal of Software Engineering & Applications (IJSEA), Vol.3, No.6, November 2012
27
[33] Bakota, T., Beszédes, Á., Gergely, T., Gyalai, M. I., Gyimóthy, T., Füleki, D. (2008). Semi-
Automatic Test Case Generation from Business Process Models. This research was supported in part
by the Hungarian national grants RET-07/2005, OTKA K-73688 and TECH_08-A2/2-2008-0089
SZOMIN08. Retrieved from http://www.inf.u-
szeged.hu/~beszedes/research/bakota09_semiautomatic.pdf
[34] Mani, S., Sinha, V. S., Sinha S. Dhoolia, P. Mukherjee, D. Chakraborty, S. (2009). Efficient Testing
of Service-Oriented Applications Using Semantic Service Stubs. IEEE International Conference on
Web Services, 197-204. Doi: 10.1109/ICWS.2009.40
[35] Yuan, Q., Wu, J., Liu, C., Zhang, L. (2008). A Model Driven Approach Toward Business Process
Test Case Generation. 10th International Symposium on Web Site Evolution (WSE), 41-44. Doi:
10.1109/WSE.2008.4655394
[36] Li, B., Qiu, D., Ji, S., Wang, D. (2010). Automatic Test Case Selection and Generation for Regression
Testing of Composite Service Based on Extensible BPEL Flow Graph. 26th IEEE International
Conference on Software Maintenance in TimiSoara, Romania, 1-10. Doi:
10.1109/ICSM.2010.5609541
[37] Li, B., Qiu, D., Leung, H., Wang, D. (2012). Automatic test case selection for regression testing of
composite service based on extensible BPEL flow graph. The Journal of Systems and Software
Volume 85, Issue 6, 13001324. Doi:10.1016/j.jss.2012.01.036
[38] Canfora, G., & Penta, M. D. (2006). SOA Testing and Self-Checking. International Workshop on
Web Services Modeling and Testing (WS-MaTe 2006), 3-12. Retrieved from
http://www.selab.isti.cnr.it/ws-mate/Canfora_WS-MaTe.pdf
[39] Zmudaa, D., Psiuk, M., Zielinski, K. (2010). Dynamic monitoring framework for the SOA execution
environment. International Conference on Computational Science (ICCS 2010), 1, 125-133.
Doi:10.1016/j.procs.2010.04.015
[40] Kabbani, N., Tilley, S., Pearson, L. (2010, April 510). Towards an Evaluation Framework for SOA
Security Testing Tools. SysCon 2010 IEEE International Systems Conference San Diego, CA, 438-
443. Doi: 10.1109/SYSTEMS.2010.5482322
[41] Ilieva, S., Pavlov, V., Manova, I. (2010). A Composable Framework for Test Automation of Service-
Based Applications. 2010 Seventh International Conference on the Quality of Information and
Communications Technology, 286-291. Doi: 10.1109/QUATIC.2010.54
[42] Cao, T.D., Felix, P., Castanet, R. (2010). WSOTF An Automatic Testing Tool for Web Services
Composition. Fifth International Conference on Internet and Web Applications and Services, 7-12.
Doi: 10.1109/ICIW.2010.9
[43] Lallali, M., Zaidi, F., Cavalli, A., Hwang, I. (2008). Automatic Timed Test Case Generation for Web
Services Composition. Sixth European Conference on Web Services, 53-62. Doi:
10.1109/ECOWS.2008.14
[44] Bertolino, A., Angelis, G.D., Lonetti, F., Sabetta, A. (2008). Automated Testbed Generation for
Service-oriented Mobile Applications. 34th Euromicro Conference Software Engineering and
Advanced Applications, 321-328. Doi: 10.1109/SEAA.2008.33
[45] Martin, E., Basu, S., Xie, T. (2007). Automated Testing and Response Analysis of Web Services.
IEEE International Conference on Web Services (ICWS), 647-654. Doi: 10.1109/ICWS.2007.49
[46] Conroy, K. M., Grechanik, M., Hellige, M., Liongosari, E. S., Xie, Q. (2007). Automatic Test
Generation From GUI Applications For Testing Web Services. Software Maintenance, IEEE
International Conference on ICSM, 345-354. Doi: 10.1109/ICSM.2007.4362647
[47] Melo A. C.V. d., & Silveira, P. (2011). Improving data perturbation testing techniques for Web
services. Information Sciences 181, 600619. Doi:10.1016/j.ins.2010.09.030
[48] Mohammad, A. F., & Mcheick, H. (2011). Cloud Services Testing An Understanding. The 2nd
International Conference on Ambient Systems, Networks and Technologies, Procedia Computer
Science, 5, 513520. Doi:10.1016/j.procs.2011.07.066
[49] Palacios, M., Garcio-Fanjul, J., Tuya, J. (2011). Testing in Service Oriented Architectures with
dynamic binding: A mapping study. Information and Software Technology, 53, 171189.
Doi:10.1016/j.infsof.2010.11.014
... The result shows that the data set can process the JSON message code, receive and transfer data specifically through HTTP, and is easy to access using JavaScript. It is more effective than SOAP which use less CPU processor power and a simpler code [10]. ...
... SOA for web service has the following: A standard for communication The same presentation pattern and data exchange The details of language which are describable A registration and searchable pattern on the web service system Therefore, SOA is an approach that has been used to design the architecture for the service. RESTful is a service which processes small functions such as data setting, data approved from clients, or other basic services [10], [11]. ...
... When service is created, users do not depend on the knowledge of the service creation. SOA exchanges data deploying protocols [10,11]. ...
Article
Full-text available
For the past few years, technology has been steadily rising and leading to rapid change in the development process. Information & Communication Technology (ICT) approaches have been moving toward new development trends with a huge elevation of comprehension. Service providers are always seeking customer satisfaction and due to multiple essentials, clients prefer to migrate from one application to another. Newer software applications are being launched, but they are constricted within a defined scope and cannot perform functions of interoperable components. Therefore,the aim of this study is to facilitate clients with multiple services over a single platform and gain knowledge about cloud computing, service computing techniques to enable flexibility and interoperability between applications. This work proposes an STA-Vision19 cloud platform and Service Integration Bridge (SIB) as solutions. To validate methods, a basic prototype for the proposed framework STA-Vision19 and two applications E-Learning and Blogging System were implemented. In results, software evaluation parameters, and testing results are discussed, which indicates that the proposed system can collaborate two or more applications and it is beneficial to use as compared to an isolated system.
... So, testing is the challenging task at this stage. Other various challenges in testing SOA are as follow [1] [2][5]-1. SOA is composed of web-service components dispersed over different hardware and operating system platform. ...
Article
Full-text available
Service-Oriented Architecture(SOA) is an approach for designing, deploying and managing services that represent reusable business functionality. SOA removes the gap between software and business. Reliability and fault-free implementation are major concern for SOA based applications. Traditional testing are no more beneficial for verifying and validating the quality of services in SOA systems. Regression testing is inevitable that is undertaken every time to provide confidence that modification do not introduce new bugs into previously validated code. In this paper we addressed the UML based regression testing method using UML use case diagram and UML activity diagram to generate a test case in the context of case study named ' Online Shopping System' .
... This paper gives valuable information pertinent to testing in SOA, but it has not discussed about dependency analysis [6]. Motlagh [7] has done a survey of testing of SOA based system. This paper has described about testing challenges of SOA based systems. ...
Conference Paper
This paper presents a survey of Dependency Analysis of Service Oriented Architecture (SOA) based systems. SOA presents newer aspects of dependency analysis due to its different architectural style and programming paradigm. This paper surveys the previous work taken on dependency analysis of service oriented systems. This study shows the strengths and weaknesses of current approaches and tools available for dependency analysis task in context of SOA. The main motivation of this work is to summarize the recent approaches in this field of research, identify major issue and challenges in dependency analysis of SOA based systems and motivate further research on this topic.
... Many other approaches have advanced the state of the art regarding automated model-driven functional testing of Web services [28,48,52,56,86], and of composition of Web services through WS-BPEL processes (see [77] for a survey), but they are limited to unit black-box testing, combined with the "assisted automation" of either test case generation or test execution, or test case selection, as currently offered in available state-of-the-art tools such as SoapUI [5], Oracle SOA Suite [69], and Parasoft [16]. In particular, to the best of our knowledge, there is no available solution for the full automation across the spectrum of the critical tasks (test input and oracle generation, test execution/arbitration, dynamic test case prioritisation, test planning) of end-to-end testing of large-scale, multi-stakeholder services architectures. ...
Article
Full-text available
This paper presents the approach to functional test automation of services (black-box testing) and service architectures (grey-box testing) that has been developed within the MIDAS project and is accessible on the MIDAS SaaS. In particular, the algorithms and techniques adopted for addressing input and oracle generation, dynamic scheduling, and session planning issues supporting service functional test automation are illustrated. More specifically, the paper details: (i) the test input generation based on formal methods and temporal logic specifications, (ii) the test oracle generation based on service formal specifications, (iii) the dynamic scheduling of test cases based on probabilistic graphical reasoning, and (iv) the reactive, evidence-based planning of test sessions with on-the-fly generation of new test cases. Finally, the utilisation of the MIDAS prototype for the functional test of operational services and service architectures in the healthcare industry is reported and assessed. A planned evolution of the technology deals with the testing and troubleshooting of distributed systems that integrate connected objects.
... Motlagh has done a survey of testing of SOA based system. Motlagh has described about testing challenges of SOA based systems [7]. Lewis, Smith and Kontogiannis [8], in their report, outlined the SOA Research Agenda. ...
Article
Full-text available
This paper presents some observations on Dependency Analysis of Service Oriented Architecture (SOA) based systems. In general, dependency analysis is based on the internal properties of artifacts (objects/components/services) and inter-relationship between the artifacts in the system. In order to make a dependency analysis for SOA based systems, one have to consider the special features of SOA that make them different with other approaches. This paper surveys the previous works taken on dependency analysis of service oriented systems. The present work provides insights about definitions related to service dependency, the modeling and analysis techniques of service dependency analysis, failure results to service dependence and some research challenges of the topic.The contribution of this paper is for novice researchers working on this topic as they can get an overview of dependency analysis of SOA based systems for their further research.
Conference Paper
Service Oriented Architecture is a hole among the software and business enterprises. SOA coordinates numerous web services to get the type of network benefits that stretches ensure about what organization administrative effort and their consequences. So, the work must be performed for a programmed experiment age for SOA based application. However, full inclusion of XML components is absent. Supposedly this everything works don't endeavours to all potential components of XML blueprint present in WSDL document. There is a need to apply various affirmations on the helping activity for creating the experiments. To conquer the issue for SOA base application, we proposed a testing Model. This model assists us with getting the programmed experiments of SOA based application. In order to develop new test model, we use our experimental determination, calculation and experimental test cases as guide. Eventually, the results of the execution of the test suite and the inclusion of XML diagram components in the WSDL file are produced.
Article
Full-text available
A SOA (Service Oriented Architecture) is an enterprise-scale IT architecture for linking resources on demand. In a SOA, resources are made available to participants in a value net, enterprise, and line of business. Service-oriented applications can be expensive to test because services are hosted remotely, are potentially shared among many users, and may have costs associated with their invocation. For interactive web services development SOA is a good approach. To find the desire outcomes from this architecture, we have to test this architecture than we can develop desire services from this architecture. There lots of works has done to test this architecture, like unit and integration testing are used to test this architecture but these methods cannot give hundred present testing capacity. In this paper we are concentrating to find a right approach to test the SOA. Regression testing is a better approach, rather than the previous methods.
Conference Paper
Nowadays complex and large-scale systems are developed for solving the problems by integrating the services. SOSE (Service Oriented Software Engineering) is the modern approach of software engineering to deal with these services. For the past 2-3 years’ services have increased exponentially, among those which is the best one for the good software development is a challenge. The detailed study of SOSE is observed with respect to the Traditional Software Engineering. SOSE gives the thorough analysis platform for the Quality of Service (QoS), which contributes to cost-efficient software development. In this Quality of service is improved by reusability and scalability of services, which leads to solving many issues, which occur during software engineering maintenance and evolution. The SOSE differs Traditional SE with the consideration of its structure, method of implementation etc. like attributes. This paper gives the crucial and clear idea about the SOSE.
Article
Full-text available
The dynamic nature of service-oriented architectures poses new challenges to system validation. Traditional testing is unable to cope with certain aspects of a service–oriented system validation, essentially because of the impossibility to test all (often unforeseen) system's configurations. On the other hand, run–time monitoring, while able to deal with the intrinsic dynamism and adaptiveness of a service–oriented system, are unable to provide confidence that a system will behave correctly before it is actually deployed. In this paper we discuss the role of testing and monitoring to validate a service–oriented system and how they can be combined to increase the confidence and reduce the cost of validation.
Article
Full-text available
Modern Information and Communications Technology (ICT) solutions are often widely distributed and highly dynamic service oriented architectures (SOA) with stringent availability requirements. Availability implies that SOA must be reconfigured, updated and maintained during runtime, while retaining their overall operational integrity. This requires that much of the adaptation, integration, configuration and testing activities typically performed offline, during development time, now have to be done online, during runtime. Current component-based runtime platforms such as SOA realize the technological foundations for runtime reconfiguration and maintenance.However, because software engineering methodology has not kept pace with the rapid leap forward in platform technology, adequate methods, techniques and tools for dealing with runtime integration and testing are not yet available.This paper discusses the industry challenges and open issues of integrating and testing SOA infrastructures during runtime.
Article
Full-text available
The Service-Oriented Architecture (SOA) is the emerging paradigm in information technology. More than a true "architecture", SOA provides a general reference model for the development, deployment and management of distributed dynamic systems. Companies are progressively adopting service-oriented technology, because of its many (real or idealized) foreseen benefits, among which notably loose coupling and dynamic interoperability. Such benefits, however, can only be achieved through discipline and standardization: in this respect, SOA Governance qualifies a framework of policies, procedures, design rules and documentation standards to be enforced for ensuring that different services and components can successfully cooperate towards a shared business goal. What about testing of such composite SOA applications? Little attention has been devoted so far by researchers to SOA testing, but awareness is raising that existing techniques and tools are not adequate. Our position in this paper is that the establishment of a test governance framework is a key issue for enabling SOA testing at integration and system levels. We discuss the inter-relation between technical and "social" aspects of SOA application testing, and attempt a first abstract definition of a SOA Test Governance notion. We provide examples of proposed SOA testing approaches that, more or less explicitly, rely on a cross-organization agreed test process.
Article
Full-text available
To generate test cases for the unit testing of business process written in BPEL, developers have to prepare input data for the BPEL process under test (PUT) and verification conditions for output data from the PUT. It could be a tedious task due to the complexity of the PUT which describes the web flow of a distributed collaboration of individual service providers executing concurrently. This paper shows how Stream X-machine (SXM) based testing techniques could be applied to automatically generate test cases for BPEL process. The SXM describes a system as a finite set of states, an internal memory and a number of transitions between the states. One of the strengths of using a SXM to specify a system is that, under certain well defined conditions, it is possible to produce a test suite that is guaranteed to determine the correctness of the implementation under test.
Article
Full-text available
Service-oriented architecture (SOA) is the latest attempt to better link the business with technology. Testing a SOA applications become more and more complex, as it should be continuous, not just in development and integration, but in deployment, because an SOA by nature is never a static application. Even if each service in a SOA application is tested thoroughly and carefully the quality of the final application may suffer because testing is not enough after the integration of the services. This article presents the challenges and problems that test teams experience when testing SOA applications. It also summarizes how the testing of SOA is carried out now and gives some ideas on further improvements.
Article
Services are highly reusable, flexible and loosely coupled components whose changes make the evolution and maintenance of composite services more complex. The changes of composite service mainly cover three types, i.e., the processes, bindings, and interfaces. In this article, an approach is proposed to select test cases for regression testing of different versions of BPEL (business process execution language) composite service where these changes are involved. The approach identifies the changes by performing control flow analysis and comparing the paths in a new version of composite service with those in the old one using a kind of eXtensible BPEL flow graph (XBFG). Message sequence is appended to XBFG path so that XBFG can fully describe the behavior of composite service. The binding and predicate constraint information added in different XBFG elements can be used for path selection and even for test case generation. Both theoretic analysis and case study show that the proposed approach is effective.
Article
Testing to ensure service QoS is a key problem for Web Service adoption. One of the key problems for software testing is designing the finite test suit tailored to test case generation efficiently. This paper focuses on Abstract Test Suite and Executable Test Suite analysis of WSDL_based Web Service and proposes automated WSDL_based Web Service testing. A XML_based test scripting language is discussed and Executable Test Suite can be derived from Abstract Test Suite automatically based on XML parser. Executable Test Suite consists of message Schema syntactic test group, Web Service interface test group, and interoperation test group. A test process is used in WSDL_based Web Service testing. The three key techniques adopted by the test process including the test case generation, test flow, and test data generation are analyzed.
Article
For large and complex composite Web services which are composed by BPEL and a set of WSDL and XSD documents, we urgently need the aid of the tools in designing and selecting test cases. However, most of mature testing tools which generate test data with the information of WSDL are used for single service while the research for composition remains in the realm of theory. So we apply the theory of data-flow testing and try to develop a testing tool for Web services composition which can automatically detects harmful data-flow anomalies and generate test paths according to all kinds of data flow coverage criteria.
Article
Web services technology offers a WS-BPEL language for business process execution. The building block of WS-BPEL is those Web services components that collaborate to realize a certain function of the business process. Applications can now be built more easily by composing existing Web services into workflows; each workflow itself is also considered a composite Web service. As with other programs, basis path testing can be conducted on WS-BPEL processes in order to verify the execution of every node of the workflow. This paper discusses the generation of the test suite for basis path testing of WS-BPEL and an accompanying tool that can be used by service testers. The test suite consists of test cases, stubs of the constituent services in the workflow, and auxiliary state services that assist in the test; these are deployed when running a test on a particular WS-BPEL. The paper presents also a testing process for service testers. A business process of a market place is discussed as a case study.
Conference Paper
Business Process Execution Language for Web Services (BPEL4WS) is a kind of concurrent programming languages with several special features that raise special challenges for verification and testing. This paper proposes a graph-search based approach to BPEL test case generation, which effectively deals with BPEL concurrency semantics. This approach defines an extension of CFG (Control Flow Graph) - BPEL Flow Graph (BFG) - to represent a BPEL program in a graphical model. Then concurrent test paths can be generated by traversing the BFG model, and test data for each path can be generated using a constraint solving method. Finally test paths and data are combined into complete test cases.