Executable Business Process Testing

With today's requirement of quickly developing dig-itization solutions, companies often use specialized workflow languages like the BPEL or BPMN 2.0, which orchestrate services along the process flow. Because process models are of critical importance to the functioning of the organization, high quality and reliability of the implementations are mandatory. Therefore, testing becomes an even more important activity in the development process. For judging the quality of developed tests, Test Coverage Metrics can be used. Current approaches to test coverage calculation for BPEL either rely on instrumentation, which is slow, or are limited to vendor-provided unit test frameworks, in which all dependent services are mocked, which limits the applicability of such approaches. Our refined approach relies on analyzing process event logs that are written during process execution. Within this article we analyze the performance characteristics of process log analysis versus the instrumentation-based approach by running an experiment with BPEL processes and their accompanying test suites developed in an industry project. According to our findings, the improved version of process log analysis is significantly faster for all scenarios.

Because executable business processes are an important and critical software asset of organizations because they control and integrate critical information systems. Thus, testing them thoroughly is a very important task within the software development process. However, failures due to implementation defects still occur in production, which in turn means that the development team needs to analyze, fix and repair the failing processes. In order to support the activities of reproducing the problem outside of the production system and to create better test cases for verifying the fixed implementation, we propose to use process mining techniques on the production process event logs to aid the support & development teams. With our approach it is possible to automatically extract a working unit test case with all partner services being mocked that can run in a development environment. Within in this paper we present the extraction algorithm, our implementation, and possible ways to integrate the tool into the support & development process.

Assessing the quality of tests for BPEL processes is a difficult task in projects following SOA principles. Since insufficient testing can lead to unforeseen defects that can be extremely costly in complex and mission critical environments, this problem needs to be addressed. By using formally defined test metrics that can be evaluated automatically by using an extension to the BPELUnit testing framework, testers are able to assess whether their white box tests cover all important areas of a BPEL process. This leads to better tests and thus to better BPEL processes because testers can improve their test cases by knowing which important areas of the BPEL process have not been tested yet.

Today more and more business processes are digitized by implementing them in specialized workflow languages like the Business Process Execution Language (BPEL) or Business Process Model and Notation (BPMN 2.0), which orchestrate services along the process flow. Because these process models are software artefacts of critical importance to the functioning of the organization, high quality and reliability of these processes are mandatory. Testing therefore becomes an important activity in the development process. Test Coverage Metrics have long been used in software development projects to assess test quality and test progress. Current approaches to test coverage calculation for BPEL either relies on instrumentation, which is slow, or is limited to vendor-provided unit test frameworks, in which all dependent services are mocked (unit tests), which limits the applicability of such approaches. Our approach relies on analyzing process event logs that are written during process execution. This approach does not require additional infrastructure and can be used in unit tests, as well as in system and integration tests. We found that our approach for measuring test coverage is not only more flexible but also faster than an instrumentation-based approach.

Testing large-scale process integration solutions is complex and cumbersome. To tackle this problem, researchers employed behavior-driven development. They used the Business Process Model and Notation language to model domain-specific test cases. These test cases can be understood by both developers and business stakeholders and can be executed automatically.

Executable Business Processes realized in WS-BPEL and BPMN2 are used more and more for automating digitalized core processes in organizations. Due to their critical nature for the organization, these processes need to be developed with high quality standards. Existing literature concentrates on testing such processes, but do not offer integration into the development lifecycle and validation with other stakeholders. Our approach is based on Test Models that allow both the easier definition of automated test cases as well as discussion with non-technical stakeholders and, thus, can be used for business process validation and process modeling support. We define a meta-model for the BPMN-based Test Models that has been validated in a case study in an industrial project.