No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
A Scalable Microservice Infrastructure for Fleet Data Management
Abstract
Modern Internet of Things solutions using edge devices produce large amounts of raw data. In order to utilize this data, it needs to be processed, aggregated, and categorized to enable decision making for management and end-users. This data management is a non-trivial task, as the computational load is directly proportional to the amount of data. In order to tackle this issue, we provide an extensible and scalable microservice architecture that can receive, normalize, and filter the raw data and persist it in different levels of aggregation, as well as for time series analysis.KeywordsBig dataTime seriesData managementData processing
ResearchGate has not been able to resolve any citations for this publication.
Abstract Sensor data quality plays a vital role in Internet of Things (IoT) applications as they are rendered useless if the data quality is bad. This systematic review aims to provide an introduction and guide for researchers who are interested in quality-related issues of physical sensor data. The process and results of the systematic review are presented which aims to answer the following research questions: what are the different types of physical sensor data errors, how to quantify or detect those errors, how to correct them and what domains are the solutions in. Out of 6970 literatures obtained from three databases (ACM Digital Library, IEEE Xplore and ScienceDirect) using the search string refined via topic modelling, 57 publications were selected and examined. Results show that the different types of sensor data errors addressed by those papers are mostly missing data and faults e.g. outliers, bias and drift. The most common solutions for error detection are based on principal component analysis (PCA) and artificial neural network (ANN) which accounts for about 40% of all error detection papers found in the study. Similarly, for fault correction, PCA and ANN are among the most common, along with Bayesian Networks. Missing values on the other hand, are mostly imputed using Association Rule Mining. Other techniques include hybrid solutions that combine several data science methods to detect and correct the errors. Through this systematic review, it is found that the methods proposed to solve physical sensor data errors cannot be directly compared due to the non-uniform evaluation process and the high use of non-publicly available datasets. Bayesian data analysis done on the 57 selected publications also suggests that publications using publicly available datasets for method evaluation have higher citation rates.
In recent years, the Internet of Things (IoT) and big data have been hot topics. With all this data being produced, new applications such as predictive maintenance are possible. Consensus self-organized models approach (COSMO) is an example of a predictive maintenance system for a fleet of public transport buses, which attempts to diagnose faulty buses that deviate from the rest of the bus fleet. The present work proposes a novel IoT architecture for predictive maintenance and proposes a semi-supervised machine learning algorithm that attempts to improve the sensor selection performed in COSMO. With the help of the Société de Transport de l’Outaouais, a minimally viable prototype of the architecture has been deployed and J1939 sensor data have been acquired.
Automatic Vehicle Location (AVL) Systems are being introduced increasingly in many major cities around the world to improve the efficiency of our road-based passenger transport systems. Satellite-based location and communication systems, particularly the Global Positioning System (GPS) have been the platform for AVL systems which are now supporting real-time passenger information (RTPI), fleet management and operations (FMOs) and public transport priorities (FTPs), to name three key applications. The process of real-time on-board bus location can result in a substantial database where the progress of the bus is stored typically on a second-by-second basis. This is necessary for the primary real-time applications such as those listed above (e.g. RTPI, FMO and PTP). In addition, it is clear that such data could have an array of 'secondary' purposes, including use off-line for improving scheduling efficiency and for automatic performance monitoring, thus reducing or removing the need for manual on-street surveys. This paper looks at these and other innovative uses of AVL data for public transport, taking the recent iBus system in London as a current example of a modern AVL/GPS application in a capital city. It describes the data architecture and management in iBus and then illustrates two further examples of secondary data use - dwell time estimation and bus performance analysis. The paper concludes with a discussion of some key data management issues, including data quantity and quality, before drawing conclusions.
The paper describes an algorithm that was developed for estimating reliable and accurate average roadway link travel times using Automatic Vehicle Identification (AVI) data. The algorithm presented is unique in two aspects. First, it is designed to handle both steady state (mean constant) and transient (varying mean) traffic conditions. In particular, the algorithm is able to track not only travel time fluctuations that are caused by recurring congestion, but also sudden changes in roadway travel times that may result from incident or other non-recurring events. Second, the algorithm can be successfully applied on segments with low levels of AVI penetration (less than 1 percent). The algorithm estimates link travel times using a robust data-filtering procedure that identifies valid observations within a sampling interval using a dynamically varying data validity window. The size of the data validity window varies as a function of the number of observations within the current sampling interval, the number of observations in the previous interval, the number of consecutive observations outside the current validity window limits, and the travel times experienced by consecutive vehicles. Application of the algorithm to two datasets of observed travel times from the San Antonio AVI system demonstrates the validity of the proposed algorithm, and in particular, its ability to track typical and sudden travel time changes in presence of low sampling rates.
This paper presents a novel holistic framework for acquiring, processing, and applying vehicle fleet test data for use in automotive research. An overview of different vehicle fleet tests is first given. The framework features a method for using smartphones and OBD data loggers applied up front to acquire specific vehicle fleet test data. The recorded data is subsequently stored in a relational database. Acceleration and other sensor and OBD data collected is processed in this database. This processing comprises data management, metadata creation, and application of a mapmatching algorithm. Various tools then analyze the stored and processed vehicle fleet test data. Different examples of results from current and ongoing research projects will suggest the framework's prospective possibilities.
