Conference PaperPDF Available

A hybrid platform for localisation sensor fusion to control large-scale infrastructures: LocON

Authors:
A hybrid platform for localisation sensor fusion to
control large-scale infrastructures: LocON
Maarten Weyn#1, Glenn Ergeerts#2, Martin Klepal*3, Alain Chambron-4, Sylvie Couronné+5, Marc Fassbinder+6
#Department of Applied Engineering, Artesis University College of Antwerp
Paardenmarkt 92, 2000 Antwerp, Belgium
1maarten.weyn@artesis.be
2glenn.ergeerts@artesis.be
*Department of Applied Engineering, Cork Institute of Technology
Cork, Ireland
3mklepal@cit.ie
-CEA-Leti, France
4alain.chambron@cea.fr
-Fraunhofer Institute for Integrated Circuits IIS
Erlangen, Germany
5cor@iis.fraunhofer.de
6fam@iis.fraunhofer.de
Abstract Localisation systems will become more and more an
important tool for controlling large-scale infrastructures.
Although singular localisation systems limit their use to a limited
number of environments and scenarios, the diverse applicability
of localisation systems and environments requires a hybrid
combination of different technologies. In this paper we combine
GPS, LPR, Wi-Fi, UWB and RFID systems in a test bed and we
describe the necessity of a standardised data protocol. We
evaluate the architecture of the fusion of different indoor and
outdoor systems. The study showed that the systems can improve
and complement each other while overcoming gaps and
shortcomings in applicability of individual systems.
I. INTRODUCTION
We currently experience a growing demand for location-
aware services in a broad range of industrial applications. The
industries demand has driven the development of various
localisation and tracking systems that are designed to suite an
individual application. While this approach has led to the fast
proliferation of some location-aware industrial services, it also
results in systems which are dominantly based on a single
technology.
However, the rapid growth of location-aware services and
their spread into new complex industrial applications poses
extra demands on localisation and tracking systems, which are
now required to operate in heterogeneous types of
environments meeting operation requirements of multiple
location-aware control applications. Therefore, the new
localisation and tracking systems have to become extensively
multi-modal based on different complementing technologies
to cope with the increasing demands.
Different researchers already demonstrated the benefits of
combining different localisation technologies [1,2], but little
research was carried out focusing on the diversity of context-
aware applications which should be driven by the localisation
systems.
LocON‟s objective is thus to streamline the combination of
single-mode localisation and tracking systems into a multi-
modal hybrid system, which is operational across
heterogeneous environments and various control location-
aware services. The concept of LocON is based on one
platform for the „plug and play”-like integration of
localisation systems in order to control large scale
infrastructures.
Typical location-aware control services at an airport are e.g.
continuous localisation of vehicles on an apron for safety and
security, localisation of workers, sub-contractors, visitors for
safety and security, location aware tasks allocation and
scheduling, and equipment and cars continuous
commissioning and collision avoidance at the apron.
The LocON test setup combines a Wi-Fi/GSM based
system[2], using signal strength measurements in buildings,
an UWB system[3] in small, but critical areas (to distinguish
security area limits), an active RFID based system using
signal runtime measurement with geometrical methods, a
distributed system using the distance information between
nodes and a GPS-EGNOS system. This configuration enables
the LocON platform to cover a widespread set of
environments.
In order to provide the integration and the cooperation of
heterogeneous radio based localisation systems a common
protocol layer described later in this abstract was studied and
developed.
The LocON platform also fuses the location data of
different localisation systems. Being built to support control
services in large environment involving a high number of
devices the fusion is designed to be scalable for managing all
incoming data in a very limited timeframe. The scalable
aspect of the fusion architecture will be detailed in the final
paper.
The remainder of this paper describes first the diversity of
the localisation systems used in LocON. Then the LocON
protocol and data stream, which makes the combination of
different systems possible, is introduced. Afterwards the
scalable fusion architecture is described. Then one of the test
runs in Faro airport are explained in experiments
II. THE LOCON SYSTEM
A. The localisation systems
The location-aware control services for airports require
location information basically about every moving car and
staff, everywhere and at any time.
Therefore, the technologies shown in Table 1 are integrated
into one LocON hybrid localisation and tracking system.
Target
Environ
ment
Typical
Range
Typical
Accura
cy
Technology /
Localisation
method
Typical
Target
Main
Drawback
Main
Advantage
IIS-
Wismit
LOS
<50m
<1m
AOA, TOF
any
LOS
requirement
small size
tags,
accuracy
LPR
LOS
<0.3m
RTOF or
TDOA
vehicle
LOS,
High power
consumption
accuracy
UWB
LOS/
NLOS
<20m
<0.1m/
<0.3m
TOA or TDOA
any
Short Range
accuracy
WLAN
Indoor
office
<100m/
<10km
1-5m/
<10m
RSSI/
Lateration &
Fingerprinting
any
Extensive
Calibration
No
dedicated
infrast.
GPS /
Egnos
Outdoor
/ Open
Sky
n/a
1-3m
TDOA
any
Open sky
Requirement
No fixed
infrast.
Table 1: Overview of systems used in LocON
B. The LocON protocol and data stream
The LocON protocol[8] drives the “hybrid-ability” of the
concept: it allows a “standardised” communication of the
heterogeneous localisation systems with the LocON platform.
Mainly, it facilitates the communication of the position data
from various tagged objects to the fusion engine and
monitoring and control services.
In order to satisfy the requirements of the different
localisation technologies on the one hand, and the need for a
consistent protocol on the other hand, a distinction was made
between common parameters and system specific parameters
in the definition of the protocol messages. Thus, the LocON
protocol defines messages that can be categorised as follows:
Systems Position Messages are split into Standard Position
Messages and Extended Data Messages. The first one is for
simple position information, i.e. they consist of the used
coordinate system, the xyz-coordinates and a quality of
location (QoL) value. The Extended Data Messages contain
more complex, system dependent information (e.g. sensor
data) to be conveyed to the platform. They can handle any
non-recurrent, position independent information like, e.g. that
a device is overheating or malfunctioning. All these protocol
messages are directed from the localisation systems to the
LocON platform.
Additionally, the LocON platform can send configuration
messages to the localisation systems (e.g. requesting an
adjustment of the update rate) and provides firmware update
commands to the nodes in the field.
Typical applications require confidence in the obtained
positions, a safe and secure transmission and a secure
authentication of the localisation systems. In order to achieve
this authentication, a handshaking process is defined in which,
based on OpenSSL, a private and public signing key is
exchanged. The data integrity is then assured by signing every
transmitted message or data packet with a unique signature
identifying the message source. This enables to detect packet
losses and data manipulation.
Figure 1: LocON protocol principles
Figure 1 shows the basic principle of the LocON protocol.
The packet frame consists of different message types i.e
Position Messages or extended Data Messages and a header to
identify the sender source. Additionally in the LocON
protocol there is the possibility to sign each frame for source
identification. The Message Area comprised all messages
regardless of which message type. In this example the
highlighted single message is a Position Message Type for
local Cartesian coordinates with the x, y, z coordinates and the
Quality of Location Indicator in millimeters. Representing the
Positions in ECEF or LLA coordinate system is also possible.
C. The middleware architecture
This section gives a high level overview of the middleware
architecture of the LocON platform, as displayed in Figure 2.
More detailed information about the platform architecture is
available in [4].
The input data received by the middleware is coming from
the different localisation subsystems who send position data
(and possibly other data) according to the LocON protocol
described in Section B.
The entry point into the platform is the Localisation System
Interface which main responsibilities are managing the
communication with the subsystems, including aspects like
authentication. The Dispatcher component will parse the
received messages and decides based on static configuration if
this message should be processed by the Localisation Engine
or if it should just be logged using the System Data Logger.
The actual processing of the location related data is done in
the Localisation Engine component which is managed by the
Localisation Engine Manager. These core components and the
scalability properties thereof are described in more detail in
the next section. The output of the platform is controlled by
the Eventbroker component. The business applications
express their interest into certain types of events by
subscribing to a (parameterized) query. The Eventbroker
manages the subscriptions and is responsible for sending the
correct data to the subscribers. Closely related to this is the
Context Data Mining component whose main task is to match
the query subscriptions predicates on top off all data streams
generated.
Subsystem Subsystem Subsystem
Localisation System
Interface
System Data Logger Dispatcher
Localisation Engine
Manager
Localisation EngineFusion State Logger
EventbrokerContext Data Mining
Business
application
Business
application
Business
application
Configuration
LocON middleware
Figure 2: LocON architecture overview
D. The scalable fusion architecture
The fusion engine receives the raw location data coming
from the different localisation systems, and estimates the most
likely position of the object. The engine is based on a
switching system which can switch between a Kalman Filter
[5], passthrough filter, particle filter [6], or adjustment filter
and uses environment, measurement and motion models.
The fusion components are the most CPU demanding
components of the platform. The platform scalability mainly
depends on the level of fusion engine scalability, while
making sure that the end-to-end latency remains on a
satisfactory level. Therefore, several instances of the fusion
engine running in parallel to keep up with the stream of
position updates are required to prevent queuing and thus
latency increase.
The main challenge lies in the management of the fusion
engine instances so that they scale virtually unlimitedly and
ideally linearly, while keeping the latency under control. A
fully stateless model where each engine instance is initiated
with all the data it needs makes it possible to distribute
instances, but this will increase the data flow. Since some
environment data is statically defined and does not change
often at run-time, the platform can safely cache this locally in
every engine instance.
An engine instance also needs the fusion state for the
specific object, which mainly consists of the particle
distribution. This data is very dynamic so a possibility is
making use of a shared data storage system which would
allow every instance to fetch the required data when needed.
Since this data is specific for a certain object instance and
different position updates of one object are not processed at
the same time, each object can be assigned to at most one
engine instance.
In the current implementation we make use of pool of
operating system threads which each run one engine instance.
A manager assigns a thread for every incoming object id and
makes sure that subsequent position updates for those objects
are handled by the correct thread. The fusion state is kept in
local memory and managed by each instance.
The manager keeps a queue per object in which the position
updates are stored. Every engine instance will pick a work
item from one of the queues it is responsible for on a round-
robin basis.
The engine components are loosely coupled from the rest of
the platform, which makes it possible to scale out to multiple
nodes and use processes on different computers instead of
threads. The cost of doing this will be increased network
traffic.
This implementation also allows executing different fusion
algorithms in parallel for every object or dependent on the
applications needs. For fast straight forward fusion only a
Kalman Filter can be used, while in parallel for example a
Particle Filter can be executed.
E. Security
A special care has been taken regarding the security aspects
of the platform. All potential threats have been analysed, and
recommendations are made to enable the highest level of
security possible.
The first level of security measure is on the infrastructure
level. The platform relies on the hosting environment level of
security. This means having the server running in a secure
data center with access control, and gateways to different
localization technologies in secured cabinets.
The second level of security deals with all the software
interfaces to the LocON system (from subsystems to platform,
platform to control services and control services to clients).
The interface between control services and clients is using
WCF configured as HTTPS, and authentication of clients
against the control services is planned. The interface between
the platform and the Control Services is based on AMQP that
provides both authentication of the parties and secure
communications. The interface between the subsystems and
the platform is the most critical as subsystems can be remote
and connected through WAN (GPS systems for example). On
top of a solution based on TLS, we have developed a light
weight secure layer that wraps the LocON protocol, providing
authentication, privacy and anti-replay.
III. EXPERIMENT
The LocON platform was installed together with the
different individual localization systems in the Faro Airport.
In this section we will discuss one user scenario, which was
demonstrated during the project. The four different engine
types where used for this test.
Figure 3 shows a screenshot of the GUI which was used
during the testing and development of the system. The GUI
shows a small part of the Faro Airport, with in blue a part of
the passenger terminal. The black circles visualize fused
objects. The colored circles are raw, unfused, data coming
from the different individual systems. Currently 2 staff
members and 4 follow-me cars are tracked.
Figure 3: Screenshot of development GUI
Figure 4 shows the path of the security person, wearing 1
WiFi tag, 1 UWB [7] tag and 2 Wismit[7] tags, in a red line.
The person started at the right side, and walked towards the
left side. This area is covered only by the WiFi system.
Nevertheless the UWB and Wismit systems still pick up the
tags signal which results in poor estimations. This should be
filtered by the LocON system. When the person reached the
left sides, he started walking outside. The area between the
two gates inside is also covered by the UWB system. From the
moment the person walked outside only one of the Wismit
systems can take over, since UWB and WiFi does not cover
the outside in this set-up.
Figure 4: Test track of Security Person
The persons kept walking straight out where a handover
between the two Wismit systems has to happen, as shown in
Figure 5. Than he walked back inside and back to his original
position.
Figure 5: Overview of the Quality of Location reported by the
different systems during the handover of the two Wismit systems.
Figure 8 till 12 show the 4 individual filter types with the data
of the security person trajectory. The black lines show the
ground truth, the red lines indicate the error between the
estimated and real positions.
The pass through filter, just forwards the location estimates
of the different systems. The systems using a pass through
filter do not compare any of the incoming measurements to
remove outliers. This is exactly what can be seen in Figure 8.
Systems which do not cover a specific area, but do receive a
signal from the tag, will estimated a position and send this to
the server. Without processing these wrong estimates are
passed through.
The Kalman filter is shown in Figure 9 and can remove
these outliers. But this filter type will still have problems
when different systems are sending conflicting positions with
an acceptable Quality of Location parameter.
The results of the particle filter are shown in Figure 10. The
particle filter can handle multimodal distributions, but needs
to estimate at the end one position which is send to the Event
Broker. So it will also suffer from conflicting incoming data.
Additional algorithms which can be incorporated in the
particle filters motion model cannot be fully exploited in this
environment. Map filtering is the process of removes
impossible particle trajectories because of walls and obstacles.
This algorithm shows to be very useful in small environments
with a lot of walls, but in large environments like an airport
the small improvement does not compensate the additional
computational penalty.
The adjustment filter is shown in Figure 11 and postulates a
linear movement which helps removing conflicting positions
coming from individual subsystems. The drawback is that the
estimation will overshoot on sudden changes in the speed or
direction of the target.
The results are combined in Figure 6. Here the 3D error of
the full trajectory is shown. The adjustment filter behaves
best, closely followed by the particle filter. The high errors of
the PassThrough Filter originate from moments when a
system which is not covering a specific area does pick up a
signal of the device in a non-covered area and uses this
information to estimate -a wrong- estimate. All errors are
higher than expected since the 3D error is shown and some
systems report a height of 0 when the device is on the floor of
that system.
Figure 6: Overall error of the different engine types with the security
person test track.
The same results are also shown in Figure 9 in a cumulative
distribution function of the 3D root mean square errors. This
graph shows a mean error of the adjustment filter of 5.09 m,
for the particle filter 5.3 m, the Kalman filter 6.67 m and the
pass through filter 9.82 m.
Figure 7: Cumulative Distribution Function of the 3D RMS
Error of the different engine types
IV. CONCLUSIONS
In this paper, we explained the need for a standardized data
format in order to enable the fusion of different heterogeneous
systems. We have shown the aspects that need to be taken into
account when designing high performance system. LocON
needs to be able to adapt to the best state and configuration
according to its actual environment and context. A modular
switch system between engines is introduced and an
experiment in a real airport environment in explained. The
adjustment filter copes best when contradicting information is
coming from different subsystems.
REFERENCES
[1] LaMarca, A., Chawethe, Y., Consolve, S., Hightower, J., Smith, I., J.,
Shon, T., Howard, J. Hughes, J., Potter, F., et al.: Place lab: Device
positioning using radio beacons in the wild. In: Gelersen, H.-W. Want,
R. Schmidt, A. (eds.) PERVASIVE 2005. LNCS, vol 3468, pp. 116-
133. Springer, Heidelberg (2005)
[2] Klepal, M., Weyn, M., Najib, W., Bylemans, I., Wibowo, S.,
Widyawan, Hantano, B.: OLS Opportunistic Localization System for
Smart Phones Devices. In Proceedings of the 1st ACM workshop on
Networking, systems, and applications for mobile handhelds,
Barcelona (2009)
[3] Talom, F.T., Denis, B., Keignart, J., Daniele, N. and Bouix, D.:
UWB Positioning Experiment in a typical Snowy Environment In:
Proceedings of 4th Workshop on Positioning, Navigation and
Communication, 2007. WPNC '07.
[4] Denis T., Ergeerts, G., Weyn M.: Schematic overview of the LocON
framework and guidelines for environmental requirements. In: LocON
Project Deliverable 3.1 (2009)
[5] Blom, H., Bar-Shalom, Y.: The interacting multiple model algorithm
for systems with Markovian switching coefficients. In: Automatic
Control, IEEE Transactions on, vol 33, no. 8, pp. 780-789 (2002)
[6] Weyn. M and Klepal M: OLS Opportunistic Localization System for
smart phones devices. In: Mobile Phones: Technology, Networks and
User Issues, Nova (2011)
[7] Niels Hadaschik: Coexistence Analysis of the Subsystems at Radio
Signal Level. In: : LocON Project Deliverable 5.4 (2010)
[8] Millner, H., Gulden, P., Weyn, M., Fassbinder, M., Casaca, A.:
Description of the LocON protocols. In: LocON Project Deliverable
4.2 (2009)
Figure 8: Error overview of pass through filter
Figure 9: Error overview of Kalman filter
Figure 10: Error overview of particle filter
Figure 11: Error overview of adjustment filter
Conference Paper
The Belgian national railway company (NMBS) is interested in localization of rail cars in their maintenance sites, which consist of both indoor and outdoor environments. Several localization technologies and methods are currently available, each with its benefits and drawbacks regarding accuracy, range, cost, power consumption, and environmental characteristics. After examining several localization systems with regards to application requirements, power and cost constraints, we found passive RFID to be the most suitable and developed a proof of concept for a real workshop environment.
Full-text available
Chapter
People are eager to locate their peers and stay connected with them. The Opportunistic Localization System (OLS) makes that simple. OLS’s core technology is an opportunistic location system based on smart phone devices. One main advantage is that it seamlessly works throughout heterogeneous environments including indoors as opposed to GPS based systems often available only outdoors under the unobstructed sky. OLS is a phone-centric localization system which grasps any location related information that is readily available in the mobile phone. In contrast to most of the competing indoor localization systems, OLS does not require a fixed dedicated infrastructure to be installed in the environment, making OLS a truly ubiquitous localization service. The latest version of OLS strives to reduce the system ownership cost by adopting a patent covered self-calibration mechanism minimizing the system installation and maintenance cost even further. OLS’s architecture migrated from the original client-server to the current service oriented architecture to cope with the increasing demands on reusability across various environments and platforms. Thus, to scale up service for a large variety of clients. The location related information used for the estimation of mobile device location are used by existing signals in the environment and can be sensed by a smart phone. The readily available information, depending on the phone capability, is typically a subset or all of the following: the GSM/UMTS signal strength, WiFi signal strength, GPS, reading from embedded accelerometers and Bluetooth proximity information. The reliability and availability of input information depends strongly on the actual character of the mobile client physical environment. When the client is outdoors under the unobstructed sky, GPS is a favorable choice typically combined with pedometer data derived from the 3D acceleration and compass measurement. When the client is in a dense urban and indoor environment the GSM/UMTS and WiFi signal strength combined with pedometer data usually performs best. If indoor floor plan layouts are available, the map filtering algorithm can further contribute to the location estimation. However, the information about the actual type of environment is not available and proper importance weighting of all input information in the fusion engine is paramount. This chapter will describe how OLS developed and particularly give insight in OLS core technology, which is the adaptive fusion of location related information that is readily available in smart phones. The experiments carried out, shows when considering all location related information for the object location estimation namely the GSM and WiFi signal strength, GPS, PDR and map filtering, the presented opportunistic localization system achieves a mean error of 2.73 m and a correct floor detection of 93%in common environments. The achieved accuracy and robustness of the system should be sufficient for most of location aware services and application, therefore having a potential of enabling truly ubiquitous location aware computing.
Full-text available
Article
An important problem in filtering for linear systems with Markovian switching coefficients (dynamic multiple model systems) is the management of hypotheses, which is necessary to limit the computational requirements. A novel approach to hypotheses merging is presented for this problem. The novelty lies in the timing of hypotheses merging. When applied to the problem of filtering for a linear system with Markovian coefficients, the method is an elegant way to derive the interacting-multiple-model (IMM) algorithm. Evaluation of the IMM algorithm shows that it performs well at a relatively low computational load. These results imply a significant change in the state of the art of approximate Bayesian filtering for systems with Markovian coefficients
Conference Paper
In this paper, we present an UWB positioning experiment performed in a typical snowy environment with a portable ultra wideband (UWB) system. The study shows the possibility of using UWB technology in a context of real-time positioning of avalanche victims. At first, the performed experiment is described. Then, the positioning algorithm based on least-squares (LS) -Taylor series expansion (TSE) is detailed. This algorithm benefits from a prior knowledge of time difference of arrival (TDOA) error biases. Finally, experimental results obtained from measurements are discussed. The obtained results show that the proposed algorithm seems to be accurate and compatible for emergency requirements (e.g. less than lm 2D-RMSE at ranges up to 30m).
Conference Paper
Location awareness is an important capability for mobile computing. Yet inexpensive, pervasive positioning—a requirement for wide-scale adoption of location-aware computing—has been elusive. We demonstrate a radio beacon-based approach to location, called Place Lab, that can overcome the lack of ubiquity and high-cost found in existing location sensing approaches. Using Place Lab, commodity laptops, PDAs and cell phones estimate their position by listening for the cell IDs of fixed radio beacons, such as wireless access points, and referencing the beacons' positions in a cached database. We present experimental results showing that 802.11 and GSM beacons are sufficiently pervasive in the greater Seattle area to achieve 20-30 meter median accuracy with nearly 100% coverage measured by availability in people's daily lives.
OLS – Opportunistic Localization System for smart phones devices [7] Niels Hadaschik: Coexistence Analysis of the Subsystems at Radio Signal Level In: : LocON Project Deliverable 5
  • M Weyn
  • M Klepal
Weyn. M and Klepal M: OLS – Opportunistic Localization System for smart phones devices. In: Mobile Phones: Technology, Networks and User Issues, Nova (2011) [7] Niels Hadaschik: Coexistence Analysis of the Subsystems at Radio Signal Level. In: : LocON Project Deliverable 5.4 (2010) [8] Millner, H., Gulden, P., Weyn, M., Fassbinder, M., Casaca, A.: Description of the LocON protocols. In: LocON Project Deliverable 4.2 (2009)
Schematic overview of the LocON framework and guidelines for environmental requirements
  • T Denis
  • G Ergeerts
  • M Weyn
Denis T., Ergeerts, G., Weyn M.: Schematic overview of the LocON framework and guidelines for environmental requirements. In: LocON Project Deliverable 3.1 (2009)
Place lab: Device positioning using radio beacons in the wild
  • A Lamarca
  • Y Chawethe
  • S Consolve
  • J Hightower
  • J Smith
  • T Shon
  • J Howard
  • J Hughes
  • F Potter
LaMarca, A., Chawethe, Y., Consolve, S., Hightower, J., Smith, I., J., Shon, T., Howard, J. Hughes, J., Potter, F., et al.: Place lab: Device positioning using radio beacons in the wild. In: Gelersen, H.-W. Want, R. Schmidt, A. (eds.) PERVASIVE 2005. LNCS, vol 3468, pp. 116133. Springer, Heidelberg (2005)
OLS-Opportunistic Localization System for Smart Phones Devices
  • M Klepal
  • M Weyn
  • W Najib
  • I Bylemans
  • S Wibowo
  • Widyawan
  • B Hantano
Klepal, M., Weyn, M., Najib, W., Bylemans, I., Wibowo, S., Widyawan, Hantano, B.: OLS-Opportunistic Localization System for Smart Phones Devices. In Proceedings of the 1st ACM workshop on Networking, systems, and applications for mobile handhelds, Barcelona (2009)
Description of the LocON protocols
  • H Millner
  • P Gulden
  • M Weyn
  • M Fassbinder
  • A Casaca
Millner, H., Gulden, P., Weyn, M., Fassbinder, M., Casaca, A.: Description of the LocON protocols. In: LocON Project Deliverable 4.2 (2009)