Conference PaperPDF Available

Highway: A domain specific language for enterprise application integration

Authors:

Abstract and Figures

Highway is a domain-specific language for implementing enterprise application integration solutions in a technology independent and functional manner. As an internal DSL developed on top of Clojure programming language, Highway uses functional programming techniques in order to simplify enterprise application integration development. In this paper we focus on abstractions and language constructs that define Highway's approach to integration. We also cover implementation of enterprise integration patterns using Highway since they represent various common situations in enterprise application integration development.
Content may be subject to copyright.
Highway: a Domain Specific Language for Enterprise
Application Integration
Vitomir Kovanovic
School of Interactive Arts and Technology
Simon Fraser University
vitomir_kovanovic@sfu.ca
Dragan Djuric
Faculty of Organizational Sciences
University of Belgrade
dragan@dragandjuric.com
ABSTRACT
Highway is a domain-specific language for implementing en-
terprise application integration solutions in a technology in-
dependent and functional manner. As an internal DSL de-
veloped on top of Clojure programming language, Highway
uses functional programming techniques in order to simplify
enterprise application integration development.
In this paper we focus on abstractions and language con-
structs that define Highway’s approach to integration. We
also cover implementation of enterprise integration patterns
using Highway since they represent various common situa-
tions in enterprise application integration development.
General Terms
Languages
Keywords
Highway, Enterprise Application Integration, Domain Spe-
cific Language
1. INTRODUCTION
One of the central activities in enterprise software develop-
ment is the continuous process of integration of various tech-
nologically heterogeneous parts into an evolving system. En-
terprise application integration (EAI) projects have a very
high failure rate due not only to technological complexity,
but because the systems being integrated are critically tied
to the business process itself. The solutions to typical prob-
lems in EAI have been continuously rediscovered and im-
proved, and the most typical solutions are encoded as EAI
patterns - descriptions of the tried and tested general solu-
tions to typical recurring problems.
However, the implementation of these patterns (solutions)
is not an easy task itself and introduces its own share of
complexity. General-purpose programming languages do not
provide internal support for enterprise application integra-
tion. Of course, it is possible to extend the grammar of a
specific language to include such support. However, extend-
ing grammars of widely used languages may not be easy and
the relevant community may not adopt it easily. An alter-
native is to develop and use special-purpose libraries, but it
always adds complexity and keeps the impedance mismatch.
Yet another alternative is to use meta-programming [9], a
craft and a process of using tools and languages for creating,
modifying, adapting, adjusting, and otherwise transform-
ing other programs. Some programming languages support
meta-programming on mainstream platforms in such a way
that developers can extend the language/platform with the
features they need and implement first-class support for en-
terprise application integration.
The result is Highway: an internal domain-specific language
for implementing enterprise application integration solutions
in a technology independent and functional manner. As an
internal DSL developed on top of Clojure programming lan-
guage Highway uses functional programming techniques in
order to simplify enterprise application integration develop-
ment.
2. THE FOUNDATIONS
2.1 Main Challenges in EAI
Even more than typical software development projects, EAI
projects have high failure rate - according to the EAI Con-
sortium[2], around 70% of EAI projects have failed to deliver
the desired goals, mainly due to the following challenges:
Enterprise integration requires changes in the company
management. It is not only about communication be-
tween software systems but also between different de-
partments and business units.
EAI systems are critical for business because of their
mediating role in the business process management.
Failures in software integration are very expensive and
represent a big threat to the stability of the business
process.
For the most part, the developers of integration solu-
tions do not have control over existing software sys-
tems. Usually they are large legacy systems that were
incorporated into the business process over the years,
using various strategies that are often incompatible.
Figure 1: Content based router pattern
Figure 2: Message filter pattern
There are various communication technologies varying
from binary socket communication, Remote Procedure
Calls(RPC), CORBA, Web services, XML, JSON and
others. The integration techniques are still in the de-
velopment and integrating applications must be able
to communicate using all of this technologies.
2.2 Enterprise Integration Patterns
Enterprise integration patterns (EIP) represent best prac-
tices and techniques for enterprise application integration
that software engineers have been discovering and using for
a long time. EI patterns are described in depth in [6], partic-
ularly message-based integration patterns that are the main
focus of this paper. Software pattern itself, is a structured
way of documenting software engineering techniques, popu-
larized by [5].
One of the most popular integration patterns, Content Based
Router shown on Figure 1, is presented here as an illustra-
tion. It is used when a system needs to dispatch the message
to different destinations based on the message content. The
solution is to use a message consumer that encapsulate the
business logic for dispatching and then sends the message to
the appropriate destination depending of the content.
Another typical pattern is Message Filter pattern shown
on Figure 2, which is used in situations when some of the
messages should be filtered and not sent to the destination.
There are many different enterprise integration patterns cov-
ering various situations such as message filtering, dispatch-
ing, re-sequencing, splitting and aggregating, delaying, load
balancing, handling failed deliveries and many more.
3. THE REQUIREMENTS OF EAI DSL
In order to properly define DSL for application integration
we must define what are the basic requirements for one such
language.
Provide abstractions and concepts specific to
the enterprise application integration in order
to simplify development of integration solutions
Language should define most important concepts so
that engineers can develop integration solutions more
easily. This is the basic requirement that allows most
basic evaluation of quality and usability of developed
DSL.
Support for transformation rules
Since integration usually involves communicating with
different software systems and defining transformation
of messages between them it is crucial to support cre-
ation of transformation rules. DSL should support ex-
pressing various different message transformations so
that the process of developing integration solution be-
comes more easier.
Language should support functional program-
ming techniques and message transformations
in a functional way
Enterprise integration is usually implemented in a se-
ries of message transformations from one participating
application service to another. For example, message
coming from some JMS queue can be converted to the
e-mail message sent to an address that is being moni-
tored by some internal application that knows how to
interpret those messages.
Transparent and unified access to the underly-
ing communication technologies
Integration logic should be the same weather in in-
volves JMS, XML, RPC or any other technology. Com-
munication technologies are always changing and they
should have minimal impact on the stability of integra-
tion software system. This would allow easier evolu-
tion of information system since individual parts of the
system can be changed without significantly affecting
integration system.
Support for Enterprise Integration Patterns
Language should allow simple and concise expression
of most patterns since they are so common and popu-
lar in practice. This criteria is also very important as
a way to evaluate expressive power of DSL since inte-
gration patterns cover various real problems and most
of application integration solutions are just different
compositions of those patterns.
4. HIGHWAY: THE MAIN CONCEPTS
Highway is a domain specific language for enterprise appli-
cation integration that helps in an easier development of
integration software.
Because EAI using messaging is easily expressed as a func-
tional transformation of messages between participating ap-
plications and their services, Highway has five language con-
structs to express these transformations:
Message is the most basic concept in Highway. Every-
thing is based around message consumption, produc-
tion and transformation. Each message is composed
of two parts; It’s body which can be of any type, de-
pending on the purpose of message and map of head-
ers where each header entry is name-value pair where
name is simple java.lang.String and value that can
be of any type.
["new widget-order"
{"amount" 1 "price" 100 "user_id" 1001}]
Figure 3: Simple Highway message
(route (from "jms:new_orders")
(router (get-header "type" (equals "widget"))
"jms:widget_inventory"
(get-header "type" (equals "gadget"))
"jms:gadget_inventory"))
Figure 4: Content based router pattern
Endpoint represents a participating application and
its services involved in the process integration. In
Highway, they are defined as strings in the form of
protocol://destination name. Such definition of End-
points is useful because it hides all communication de-
tails from the developers, who do not need to write
low level communication code. It is very similar to the
concept of Wrapper described in [4] but also more ab-
stract since it is on the language runtime to provide
support for various communication technologies.
Route is any transformation of messages between two
or more Endpoints. As message transformation is at
the heart of message-based EAI, Route is the most
important concept in Highway and most of the DSL
is focused around easier route creation. Typical route
will read from one endpoint, transform the data and
then based on some business logic transfer that data
to the some other endpoint. The notion of route as a
basic element of EAI is very similar to the concept of
Flow described in [4].
Predicate is the function that receives message as an
argument and returns boolean value. It is very simple,
yet very powerful construct used in every functional
programming language as an input to the generic func-
tions such as filter function.
Processor is the function that receives message as an
input and also returns a message as an output. It is the
most general processing component that can be used
to implement various different message transformation
such as Message Enrichment, Filtering, Normalizing
and other transformations described in [6].
Highway is implemented as an internal DSL on top of Clo-
jure programming language. The decision to use Clojure
is justified with its meta-programming capabilities and also
its minimalist syntax. Highway built-in DSL constructs are
based in Apache Camel integration engine which is also
used as an underlying integration technology. Additionally,
GuaranaDSL’s Wrappers and Processes are very similar to
the Highway’s Endpoints and Routes[4].
Figure 4 shows typical example of content based router pat-
tern using Highway DSL. Every route is constructed using
route macro which accepts route definition elements as its
arguments. In this example router function that is used
(route (from "jms:new_orders")
(get-header "type" (equals "widget"))
(to "jms:widget_orders"))
Figure 5: Message filter pattern
RouteBuilder builder = new RouteBuilder() {
public void configure() {
from("jms:new_orders").choice()
.when(header("type").isEqualTo("widget"))
.to("jms:widget_inventory")
.when(header("type").isEqualTo("gadget"))
.to("jms:gadget_inventory");
}
};
Figure 6: Apache Camel implementation of content
based router pattern
to construct conditional routing needed for message routing.
Message is being transferred from JMS queue new_orders to
the queues gadget_inventory and widget_inventory based
on the content of message header type. This is very simple
example but still it is very common in the real software sys-
tems.
Message Filter pattern (figure 5), is another example of
Highway for popular EI pattern. JMS queue new_orders re-
ceives both Widget and Gadget orders and this route filters
only widget orders and transfers them to the JMS queue
widget_orders using the Highway’s built-in filter func-
tion.
5. EVALUATION
5.1 Highway vs. Camel
In order to evaluate Highway we will compare its main fea-
tures with Apache Camel since it is one of the most popular
EAI technologies and also very mature software product.
Apache Camel uses fluent interface[3] API design style and
allows creation of EAI systems using Java language. High-
way is on the other hand built on top of Clojure and hence
supports Clojure’s advanced features such as macros and us-
age of high order functions. Since Highway internally uses
Camel they are mostly of the same capabilities but Highway
code is much more shorter and concise due to the functional
programming style. This is the biggest overall difference
between Camel and Highway. Java as a platform for imple-
menting EAI solutions has severe limitations due to its lack
of support for functional programming techniques.
In figures 4 and 5 we have shown Highway’s implementa-
tion of two very popular patterns. In figures 6 and 7 are
shown corresponding implementations using Camel’s fluent
Java interface. As we can see Camel’s implementation is
much more verbose and do not correspond to the usual way
of writing Java programs. Also purpose of various meth-
ods is not clear just by looking at their names and docu-
mentation. They need to be looked in the context of DSL
expression and not traditional Java API browsing style.
RouteBuilder builder = new RouteBuilder() {
public void configure() {
from("jms:new_orders")
.filter(header("type").isEqualTo("widget"))
.to("jms:widget_orders");
}
};
Figure 7: Apache Camel implementation of message
filter pattern
If we compare Highway and Camel according to the require-
ments mentioned in Section 3 we can see that they both meet
all of the requirements except that Camel solution do not
allow usage of functional programming techniques for EAI
development. Also Highway’s code is a little bit shorter due
to the flexible nature of Lisp inspired syntax.
5.2 Does it Meet the Requirements?
In Section 3 we have mentioned requirements that good DSL
language for EAI should satisfy. Here we take a closer look
how Highway stands against those requirements.
Provide abstractions and concepts specific to
the enterprise application integration
As we mentioned in Section 4, Highway provides very
various concepts specific to the domain of EAI. These
concepts define most important elements that compose
any enterprise integration system. Some of these con-
cepts are found in other integration solutions such as
GuaranaDSL’s Processes, Wrappers and Tasks[4] and
some, such as Predicates, are common in the functional
programming world.
Support for transformation rules
Highway main goal is to support creation of transfor-
mation rules. Programs in Highway are mostly consist-
ing of definitions of Routes and each route represents
one transformation rule that connects two or more
endpoints in a business process. Route abstraction is
very similar to the GuaranaDSL’s IntegrationLink[4]
abstraction and RouteDefinition concept in Apache
Camel[8, 7].
Support for functional programming
Since Highway is internal DSL on top of Clojure, it has
all functional capabilities of its host language that en-
able usage of functional principles in the process of de-
signing integration solutions. This highly affects over-
all usability of the language since it is possible to create
highly abstract transformations as higher order func-
tions that can be easily specialized to meet different
specific details.
Transparent and unified access to the underly-
ing communication technologies
Highway uses the same syntax as Apache Camel for
defining access to the communication technology. It
is based on the mature URI[1] standard and it allows
unambiguous definition of different resources in the en-
terprise system.
Support for Enterprise Integration Patterns
Highway support most of the integration patterns ex-
plained in the [6]. Many of the patterns such as“Mes-
sage Channel”, “Message” and “Message Endpoint” are
integrated in the language design and therefore there is
no special effort required to implement those patterns.
6. CONCLUSIONS
DSL proposal presented in this paper contributes to im-
proving development of integration solutions by providing
basic concepts and building blocks for application integra-
tion development. Biggest change in comparison to the
existing EAI tools and languages is that Highway enables
functional programming in the context of EAI development.
This is very important since functional programming tech-
niques provide abstractions that are very useful in the con-
text of EAI and can help reducing time and costs of in-
tegration solution development. Also Highway allows sim-
ple and consistent way for accessing various communication
technologies. As enterprise system evolves over time, tech-
nologies used to communicate between software systems are
also changing and we need a systematic approach to embrace
those changes.
Highway enables expressing most of the enterprise integra-
tion patterns that are common in practice. They represent
proven designs and also common vocabulary among appli-
cation integration practitioners and by providing support
for integration patterns Highway contributes to minimizing
software engineering effort to produce integration solutions
and also standardizes integration development process.
7. REFERENCES
[1] T. Berners-Lee, R. Fielding, and L. Masinter. Uniform
resource identifiers (uri): generic syntax, 1998.
[2] EAI Consortium. Avoiding eai disasters, 2004.
Retrieved March 15, 2010, from Information
Management
www.information-management.com/news/8086-1.html.
[3] M. Fowler and R. Parsons. Domain-specific languages.
Addison-Wesley Professional, 2010.
[4] R.Z. Frantz, A.M.R. Quintero, and R. Corchuelo. A
domain-specific language to design enterprise
application integration solutions. International Journal
of Cooperative Information Systems (IJCIS),
20(2):143–176, 2011.
[5] E. Gamma, R. Helm, R. Johnson, and J. Vlissides.
Design patterns: elements of reusable object-oriented
software, volume 206. Addison-wesley Reading, MA,
1995.
[6] G. Hohpe and B. Woolf. Enterprise integration
patterns: Designing, building, and deploying messaging
solutions. Addison-Wesley Professional, 2004.
[7] C. Ibsen and J. Anstey. Camel in Action. Manning
Publications Co., 2010.
[8] P. Kolb. Realization of eai patterns with apache camel.
Student Research Project, Universit¨
at Stuttgart,
Germany, 2008.
[9] T. Sheard. Accomplishments and research challenges in
meta programming. Lecture Notes in Computer
Science, 2196:2–44, 2001.
... A DSL called Guaraná was proposed for design and automatic deployment of integration solutions. Another DSL for EAI called Highway is presented in [9]. It is based on Apache Camel and Clojure. ...
Conference Paper
Full-text available
Mapping elements of various interfaces is one of the most complex tasks in enterprise integration. Differences in the ways that these interfaces represent data in lead to the need of conflict detection and resolving. We present an approach where a structural model of the interfaces can be annotated with a semantic model and used together to (semi-)automate this process. A domain-specific language (DSL) is proposed that can be used to specify criteria for interface element mapping, define conflicts with steps for their resolution if possible, and how the resulting mappings will be translated into expressions needed for code generation. This DSL is intended to give the user the possibility to customise a prototype tool (which we have presented earlier) enabling us to practically test our approach and yield a real-world runnable implementation. Code generated by this tool is deployable to an enterprise service bus (ESB).
... Following Model-Driven Architecture and using a DSL was proposed in Sleiman et al. (2009) with a DSL called Guarana for design and automatic deployment of integration solutions. Another DSL for enterprise integration called Highway is presented in Kovanovic and Djuric (2012). It is based on Apache Camel and Clojure. ...
Article
Full-text available
In enterprise integration, one of the most complex tasks is to map elements of various interfaces to each other. These interfaces often transport data in different ways. This means that some form of data transformation is needed. We present an approach where structural and semantic models of the interfaces can be used together to automate or semi-automate this otherwise tedious and error prone manual process. Some of the possible criteria for interface element mapping are shown, along with semantic conflicts and how they are detected and resolved. We also present a prototype tool, including an overview of its architecture, that enables us to test our approach and have a real-world runnable implementation that is deployable on an enterprise service bus runtime. Finally, we show how some of the steps in the mapping and conflict resolution process could be made configurable by the user, making the integration developer agnostic with respect to the technical implementation of the involved systems.
Article
Full-text available
This article addresses major information systems integration problems, approaches, technologies, and tools within the context of Model-Driven Software Engineering. The Guaraná integration platform is introduced as an innovative platform amongst state-of-the-art technologies available for enterprises to design and implement integration solutions. In this article, we present its domain-specific modelling language and its industrial cloud-based web development platform, which supports the design and implementation of integration solutions. A real-world case study is described and analysed; then, we delve into its design and implementation, to finally disclose ten measures that empirically help estimating the amount of effort involved in the development of integration solutions.
Article
Data transformation is a key task in mashup development (e.g., access to heterogeneous services, data flow). It is considered as a labour-intensive and error-prone process. The possibility of reusing previously specified mappings promises a significant reduction in manual and time-consuming transformation tasks, nevertheless its potential has not been fully realized in current approaches and systems. In this chapter, we study the problem of data transformation logic reuse in mashup development platforms. We formulate the problem and propose a solution that features novel reuse abstractions and techniques including spreadsheet templates, mapping generalization, and similarity join. Given a spreadsheet instance that is being mapped to the target schema, we recommend a list of mapping formulas that can be potentially reused for the instance. We implemented a prototype of the proposed solution and evaluated its performance via synthetic datasets. © 2014 Springer Science+Business Media New York. All rights are reserved.
Article
Communication between heterogeneous systems is difficult to be achieved without affecting the existing systems in Enterprise Application Integration. A kind of dynamic web services publishing software based on CXF is proposed in this paper. The purpose of this software are twofold. Firstly, it encapsulates specified data called by other systems into web service published dynamically which unifies customization among heterogeneous systems. Secondly, the function of service monitoring provided allows users to get the real-time information of published web services. The proposal software has been deployed in a Chinese large-scale military enterprise. Practice in the enterprise shows that the software can simplify web service development, quickly build a web service which encapsulates large amounts of data, and achieve information sharing within the enterprise environment.
Article
Full-text available
Enterprise Application Integration (EAI) solutions cope with two kinds of problems within software ecosystems, namely: keeping a number of application's data in synchrony or creating new functionality on top of them. ESBs provide the technology required to implement a variety of EAI solutions at sensible costs, but they are still far from negligible. It is not surprising then that many authors are working on proposals to endow them with domain-specific tools to help software engineers reduce integration costs. In this article, we introduce a proposal called Guaraná. Its key features are as follows: it provides explicit support to devise EAI solutions using enterprise integration patterns by means of a graphical model; its DSL enables software engineers to have not only the view of a process, but also a view of the whole set of processes of which an EAI solution is composed; both processes and tasks can have multiple inputs and multiple outputs; and, finally, its runtime system provides a task-based execution model that is usually more efficient than the process-based execution models in current use. We have also implemented a graphical editor for our DSL and a set of scripts to transform our models into Java code ready to be compiled and executed. To set up a solution from this code a software engineer only needs to configure a number of adapters to communicate with the applications being integrated.
Article
69 pages Integrating enterprise applications is a challenging topic. The book "Enterprise Integration Patterns" by Gregor Hohpe and Bobby Woolf describes a collection of patterns residing in the domain of enterprise application integration using messaging. The book describes frequently recurring problems in this domain and their solution in the form of patterns. The main objective of this work is to analyze the implementation of these Enterprise Integration Patterns in Apache Camel. Apache Camel is a routing and mediation engine which implements the Enterprise Integration Patterns. It provides a framework where these patterns can be used to integrate different applications. In this work the Apache Camel platform is evaluated regarding the correct implementation of the Enterprise Integration Patterns. The result of this evaluation is then used to develop a framework, which extends the Apache Camel framework, enabling the execution of parameterized patterns. Additionally an existing graphical editor for the modeling of integration scenarios using patterns is extended. In the existing editor these modeled integration scenarios can be used to generate corresponding WS-BPEL processes. The extension implemented in this work additionally allows to generate Apache Camel based Java applications executing the modeled integration scenarios.
Article
The book is an introduction to the idea of design patterns in software engineering, and a catalog of twenty-three common patterns. The nice thing is, most experienced OOP designers will find out they've known about patterns all along. It's just that they've never considered them as such, or tried to centralize the idea behind a given pattern so that it will be easily reusable.
Article
A Uniform Resource Identifier (URI) is a compact string of characters for identifying an abstract or physical resource. This document defines the generic syntax of URI, including both absolute and relative forms, and guidelines for their use; it revises and replaces the generic definitions in RFC 1738 and RFC 1808.
Conference Paper
this paper into several sections. As an overview, in Section 2, I try and classify meta-programs into groups. The purpose of this is to provide a common vocabulary which we can use to describe meta-programming systems in the rest of the paper
Avoiding eai disasters
  • Eai Consortium
EAI Consortium. Avoiding eai disasters, 2004. Retrieved March 15, 2010, from Information Management www.information-management.com/news/8086-1.html.