No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Maritime Traffic Monitoring Based on Vessel Detection, Tracking, State Estimation, and Trajectory Prediction
Abstract
Maneuvering vessel detection and tracking (VDT), incorporated with state estimation and trajectory prediction, are important tasks for vessel navigational systems (VNSs), as well as vessel traffic monitoring and information systems (VTMISs) to improve maritime safety and security in ocean navigation. Although conventional VNSs and VTMISs are equipped with maritime surveillance systems for the same purpose, intelligent capabilities for vessel detection, tracking, state estimation, and navigational trajectory prediction are underdeveloped. Therefore, the integration of intelligent features into VTMISs is proposed in this paper. The first part of this paper is focused on detecting and tracking of a multiple-vessel situation. An artificial neural network (ANN) is proposed as the mechanism for detecting and tracking multiple vessels. In the second part of this paper, vessel state estimation and navigational trajectory prediction of a single-vessel situation are considered. An extended Kalman filter (EKF) is proposed for the estimation of vessel states and further used for the prediction of vessel trajectories. Finally, the proposed VTMIS is simulated, and successful simulation results are presented in this paper.
... In this increasingly complex navigation environment, ship trajectory prediction can provide information and reference for ship avoidance decisions, which is a key skill to improve ship perception and ensure ship safety. Therefore, improving the accuracy of ship trajectory prediction is of great significance for ship collision avoidance, intelligent navigation, and collision crisis warning [8][9][10][11][12][13][14][15]. ...
... The ship trajectory prediction problem is a nonlinear problem, and for the nonlinear problem, a finite element scheme is a good tool [16]. At the early stage of trajectory prediction research, most scholars used the prediction of ship trajectories based on the construction of physical motion models, including curve models [13], lateral models [17], and ship models [3]. The physical model describes the ship operation model by analyzing the problem and establishing a series of mathematical formulas, taking into account the influence of various factors during ship navigation as much as possible. ...
... Many scholars choose neural networks as a method for ship trajectory prediction because of their powerful nonlinear fitting ability and parallel computing capability [21], and these prediction methods are becoming more and more popular in the field of ship navigation [15,22,23]. Perera et al. [13] proposed an artificial neural network (ANN) for ship trajectory prediction combined with an extended Kalman filter for ship state prediction. Ma et al. [24] proposed a 4D trajectory prediction model based on BP neural network for the problem that traditional trajectory prediction methods cannot meet high accuracy, multidimensionality, and real-time. ...
Nowadays, maritime transportation has become one of the most important ways of international trade. However, with the increase in ship transportation, the complex maritime environment has led to frequent traffic accidents, causing huge economic losses and safety hazards. For ships in maritime transportation, collision avoidance and route planning can be achieved by predicting the ship’s trajectory, which can give crews warning to avoid dangers. How to predict the ship’s trajectory more accurately is of great significance for risk avoidance. However, existing ship trajectory prediction models suffer from problems such as poor prediction accuracy, poor applicability, and difficult hyperparameter design. To address these issues, this paper adopts the Bidirectional Long Short-Term Memory (BILSTM) model as the base model, as it considers contextual information of time-series data more comprehensively. Meanwhile, to improve the accuracy and fitness of complex ship trajectories, this paper adds an attention mechanism to the BILSTM model to improve the weight of key information. In addition, to solve the problem of difficult hyperparameter design, this paper optimizes the hyperparameters of the Attention-BILSTM network by fusing the Whale Optimization Algorithm (WOA). In this paper, the AIS data are filtered, and the trajectory is complemented by the cubic spline interpolation method. Using the pre-processed AIS data, this WOA-Attention-BILSTM model is compared and assessed with traditional models. The results show that compared with other models, the WOA-Attention-BILSTM prediction model has high prediction accuracy, high applicability, and high stability, which provides an effective and feasible method for ship collision avoidance, maritime surveillance, and intelligent shipping.
... The geographical areas of the grid cells were the same at the same latitudes and the geographic area of the grid cells at the same longitudes decreased with increasing latitude. According to each specific grid area, the fishing duration of the canvas stow net vessels per square kilometre was calculated for each month and the results were taken as the fishing intensity of the canvas stow net fishing vessels (Perera et al., 2012;Murray et al., 2013). Fig. 5 shows the spatial distribution of the fishing intensity of canvas stow net vessels in each production month of 2018 in the East China Sea and the Yellow Sea. ...
... The geographical size of the 0.01°×0.01° grid in the WGS1984 coordinate system was equivalent to a 1 × 1 km grid (Perera et al., 2012;Murray et al., 2013). The use of a 0.01°×0.01° ...
Present study used the position data of BeiDou Vessel Monitoring System (VMS) in 2018, with respect to motorised fishingvessels in the East China Sea and the Yellow Sea to construct a fishing vessel operating status classification model based onthreshold, deep neural network and DBSCAN density clustering algorithm. The geographic grid was divided into cells of0.1°×0.1° and the average fishing time per square km (h km-2) in each grid was calculated to obtain the spatial distributionof fishing intensity in the study region in 2018. The results showed that the threshold method could classify fishing vesselsailing, anchoring and other states with an accuracy of more than 95%. The deep neural network and DBSCAN algorithmcould classify the two states of netting and closing with an accuracy of 94.7%. By classifying the status of fishing vessels,quantitative monitoring can be carried out to better serve the management of marine fishery resources and marine ecologicalprotectionKeywords: China, DBSCAN, Deep neural network, Fishing intensity, Spatial distribution, VMS, Voyage extraction
... First, the frequency of AIS data updates depends on the navigation status of the ship and varies from several seconds to a few minutes. Second, there may be many missed ship trajectory points due to limited communication bandwidth, high data loss rate, and sensor errors [3]. Both issues may lead to incomplete and inaccurate ship movement, which causes great difficulties for real-time maritime surveillance. ...
... However, this method relies on the historic motion pattern data without anomaly information and is not suitable for actual situations of ship movements. Perera et al. [3] propose an extended Kalman filter method to predict ship trajectory by adding estimated noise in the kinematic model. The Kalman filter method proposes to solve the problem of missing points of ship trajectory through a polynomial. ...
The prediction of ship location has become an increasingly popular research hotspot in the field of maritime transportation engineering, which benefits maritime safety supervision and security. Existing methods of ship location prediction based on motion characteristics have a large uncertainty and cannot guarantee trajectory prediction accuracy of the target ship. An improved method of location prediction using k-nearest neighbor (KNN) is proposed in this paper. An expanded circle area of the latest point of the target ship is first generated to find the reference points with similar movement characteristics in the constraints of distance and time intervals. Then, the top k-nearest neighbors are determined based on the degree of similarity. Relationships between the reference point of each neighbor and the latest points of the target ship are calculated. The predicted location of the target ship can then be determined by a weighted calculation of the locations of all neighbors at the predicted time and their relationships with the target ship. Experiments of ship location prediction in 10 min, 20 min, and 30 min were conducted. The correlation coefficient of the location prediction error for the three experiments was 0.992, 0.99, and 0.9875, respectively. The results show that ship location prediction with reference to multiple nearest neighbors with similar movements can provide better accuracy.
... The drawback of [4], [5] is that the parameters are assumed constant. Kalman filter (KF) and Extented KF approaches are applied in [6] to estimate ship trajectories to be utilized in collision avoidance. Large number of sensor measurements are required for [6]. ...
... Kalman filter (KF) and Extented KF approaches are applied in [6] to estimate ship trajectories to be utilized in collision avoidance. Large number of sensor measurements are required for [6]. ...
... Routes are then represented through a model, which can be improved through the inclusion of waypoints (harbors, offshore platforms, and entry and exit points in the area). Examples of trajectory-based algorithms exploit extended Kalman filter [15], similarity-based approach and kernel-based machine learning methods [16], synthetic route knowledge [17], a data-driven nonparametric Bayesian model based on a Gaussian Process [18], neural networks [19], deep learning [20], and generative models [21]. ...
... Discretization was carried out by dividing the course into 8 clock faces, as illustrated by All the possible speeds were grouped into 4 slots (Table 2): slow, medium, high, and very high [15]. Finally, latitude and longitude were also discretized, by setting the latitude to the value of the row in the matrix, representing the AoI, and the longitude to the column. ...
Ship Route Prediction (SRP) is an algorithm that allows assessing the future position of a ship using historical data, extracted from AIS messages. In an SRP task, it is very important to select the set of input features, used to train the model. In this paper, we try to evaluate if time-dependent features are relevant in an SRP model, based on a K-Nearest Neighbor classifier, through a practical experiment. In practice, we build two models, with and without the Date Time features, and for both models, we calculate some performance metrics and the SHAP value. Tests show that although the model with the Date Time features outperforms the other model in terms of evaluation metrics, it does not in the practical experiments.
... Each particle was sampled from a probability density function and given a likelihood weight (Xiao et al., 2019). The particle filtering method has been applied to traffic forecasting in the maritime traffic domain and trajectory prediction (Perera et al., 2012;. The particle filtering approach is based on the traffic patterns recovered, and has two parts: 1) motion pattern extraction; 2) motion forecasting. ...
... The control theory-based models mainly include the Extended Kalman Filter (EKF) model and Linear Quadratic Regulator (LQR) model in maritime trajectory prediction.The EKF has been used for ship tracking and trajectory prediction. Compared with the standard Kalman filter method, the EKF can be used for non-linear models through linearisation system error and many prediction models were based on the EKF(Perera et al., 2010(Perera et al., , 2012Sabet et al., 2016;Juraszek et al., 2019). Perera et al. (2010 designed a maritime surveillance system that included multi-functional aspects (e.g., ship detection and tracking, manoeuvering state estimation and prediction). ...
Due to significant advances in robotics and transportation, research on autonomous ships has attracted considerable attention. The most critical task is to make the ships capable of accurately, reliably, and intelligently detecting their surroundings to achieve high levels of autonomy. Three deep learning-based models are constructed in this thesis to perform complex perceptual tasks such as identifying ships, analysing encounter situations, and recognising water surface objects. In this thesis, sensors, including the Automatic Identification System (AIS) and cameras, provide critical information for scene perception. Specifically, the AIS enables mid-range and long-range detection, assisting the decision-making system to take suitable and decisive action. A Convolutional Neural Network-Ship Movement Modes Classification (CNN-SMMC) is used to detect ships or objects. Following that, a Semi- Supervised Convolutional Encoder-Decoder Network (SCEDN) is developed to classify ship encounter situations and make a collision avoidance plan for the moving ships or objects. Additionally, cameras are used to detect short-range objects, a supplementary solution to ships or objects not equipped with an AIS. A Water Obstacle Detection Network based on Image Segmentation (WODIS) is developed to find potential threat targets. A series of quantifiable experiments have demonstrated that these models can provide reliable scene perception for autonomous ships.
... To filter noise and disturbances out of tracking data, various Kalman filtering algorithms were used. For example, in monitoring maritime traffic [12], an extended Kalman filter (EKF) obtains state estimation and predicts the navigation path with sufficiently small errors. In [13], a hardware architecture was proposed for tracking moving objects by implementing the KF on an FPGA board for high-speed optimizations: twice as fast as predecessors. ...
A well-known problem of visual object tracking is the difficulty of accurately estimating the object trajectory under conditions of environmental disturbances in the bounding box (BB) of a video camera. In this paper, we consider BB variations as Gaussian-Markov colored measurement noise (CMN). In order to perform accurate tracking in the presence of CMN, we use measurement differencing and develop a robust general unbiased finite impulse response (GUFIR) filter and use the general Kalman filter (GKF) as a benchmark. The GUFIR and GKF algorithms are tested by the “Car4" benchmark. It is shown that, in terms of the tracking precision and under the heavy disturbance with the 0.65 ⩽ Ψ ⩽ 0.95 coloredness factor, the best tracking performance is achieved using the robust GUFIR filter. When Ψ < 0.6, the GUFIR and GKF algorithms perform near equally. In the extreme point of Ψ = 1.0, where the Gauss-Markov CMN loses the stationarity, both algorithms provide zero precision and become inefficient. In general, it is concluded that the GUFIR filter, which ignores any zero mean disturbance and initial values, is much more suitable for applications in visual object tracking than Kalman-like algorithms relying on complete object information.
... Dalsnes et al. (2018) presented a GMMs approach to predict vessel positions 5-15 min into the future using AIS data. Perera et al. (2012) proposed an extended Kalman filter (EKF) for the estimation of vessel states and further used for the vessel trajectory prediction. ...
Vessel trajectory prediction is a critical aspect of ensuring maritime traffic safety and avoiding collisions. The long short-term memory (LSTM) network and its extensions have represented powerful ability of vessel trajectory prediction. However, the previous studies often did not take dynamic interactions between neighboring vessels into account. Additionally, in complex traffic conditions, trajectory prediction will acquire uncertainty, and these potential negative factors can limit the prediction of future trajectory. To enhance the prediction performance, we propose an interactive vessel trajectory prediction framework (i.e., QSD-LSTM) based on LSTM, which is embedded with the quaternion ship domain (QSD). The QSD is beneficial for avoiding unwanted collision between neighboring vessels. In addition, the operation of trajectory clustering is further incorporated into our trajectory prediction framework, potentially leading to more robust prediction results. Numerous experiments have been implemented on realistic automatic identification system (AIS)-based vessel trajectories to compare our QSD-LSTM with several state-of-the-art prediction methods. The prediction results have demonstrated the superior performance of our method in terms of both quantitative and qualitative evaluations.
... This kind of study would not exist without the availability of big data on weather conditions, including waves and wind, as well as the detection of ship performance using smart meter data. Therefore, the data can be analysed and trained using machine learning techniques toward improving the existing design and the operation of the fleet as well as ensuring safety (Moreira et al., 2021;Perera et al., 2011Perera et al., , 2012Perera et al., , 2015Mo, 2016, 2017;Uyanık et al., 2020Uyanık et al., , 2021Xu et al., 2023), as well as developing a non-intrusive load monitoring (NILM) approach to be able to continuously detect the ship loads among her life (Bucci et al., 2021;Lindahl et al., 2018;Ma et al., 2021b;Ramsey, 2004). Finally, life cycle assessment becomes an essential study that evaluates the ship propulsion systems and the fuel used, particularly the new fuels according to the ship routes in comparison to the current one. ...
This paper presents a comprehensive review of the current regulations and the various technologies as well as the decision support methods for each technology the maritime industry considers to ensure the fleet's sustainability. It covers the period between 2010 and 2022, emphasizing the last four years. It shows the impact of each technology on the reduction of ship resistance and the energy required on board, affecting the amount of fuel consumption and avoiding the transportation of harmful species around the world to achieve a smooth transition towards green shipping by improving the fleet's energy efficiency and achieving the goals of the 2050 plan. The paper covers five main topics: hull design, propulsion systems, new clean fuels and treatment systems, power systems and ship operation; and each topic has different technologies included. This study's findings contribute to mapping the scientific knowledge of each technology in the maritime field, identifying relevant topic areas, visualising the links between the topics, and recognising research gaps and opportunities. This review helps to present holistic approaches in future research supporting the cooperation between maritime industry stake-holders to provide more realistic solutions toward sustainability.
... Researchers proposed a large variety of models for future ship position prediction. For example, the extended Kalman filter was used to estimate ship motion states and predict the trajectory of a vessel by means of a kinematic model [Perera et al., 2012;Sutulo et al., 2002;Perera, 2017]. Li et al. leveraged the Support Vector Machine to predict the heave motion under the impact of waves [Li et al., 2016]. ...
Model uncertainty is pervasive and inherent in the engineering field. It could bring potential risks in real applications, especially for ship behaviour prediction under environmental disturbances. The evaluation and quantification of model uncertainty are of importance for accurate ship motion prediction. This study applies model uncertainty analysis and sensitivity analysis methods to evaluate the ship motion model's level of uncertainty against environmental disturbances and ship manoeuvres. Firstly, three models are created based on the a dynamical model (Mariner) in Marine Systems Simulator. After that, models are tested on various predefined scenarios. The similarity of predicted tra-jectories and the reference is evaluated by Euclidean distance and used to quantify the uncertainty of models. Next, statistical analysis is used to analyze the uncertainty of models. Sensitivity analysis (SA) method called 'PAWN' and 'UnivariateSpline' interpolation technology are combined to identify which factors contribute the most to model's performance. The results suggest that the uncertainty caused by external factors varies from different models under different manoeuvres. SA can tell us which factors (wind angle, wind velocity, and surge speed) have a large influence to the model uncertainty given a ship maneuver. Such analyses, on the one hand, contribute operators to choosing the optimum model according to the current conditions for better ship motion prediction. On the other hand, they can pick up the most important factors for fast uncertainty modelling.
... Additionally, they are not capable of processing long sequences and unraveling the spatial and temporal dependencies present in sequential observations. Hybrid approaches, on the other hand, combine physics-based models and machine learning models or different machine learning models to enhance the quality of the trajectory tracking process [29,30,31,32,33,34]. These approaches, however, are not free from the limitations imposed by the physics-based and machine learning models. ...
In marine surveillance, distinguishing between normal and anomalous vessel movement patterns is critical for identifying potential threats in a timely manner. Once detected, it is important to monitor and track these vessels until a necessary intervention occurs. To achieve this, track association algorithms are used, which take sequential observations comprising geological and motion parameters of the vessels and associate them with respective vessels. The spatial and temporal variations inherent in these sequential observations make the association task challenging for traditional multi-object tracking algorithms. Additionally, the presence of overlapping tracks and missing data can further complicate the trajectory tracking process. To address these challenges, in this study, we approach this tracking task as a multivariate time series problem and introduce a 1D CNN-LSTM architecture-based framework for track association. This special neural network architecture can capture the spatial patterns as well as the long-term temporal relations that exist among the sequential observations. During the training process, it learns and builds the trajectory for each of these underlying vessels. Once trained, the proposed framework takes the marine vessel's location and motion data collected through the Automatic Identification System (AIS) as input and returns the most likely vessel track as output in real-time. To evaluate the performance of our approach, we utilize an AIS dataset containing observations from 327 vessels traveling in a specific geographic region. We measure the performance of our proposed framework using standard performance metrics such as accuracy, precision, recall, and F1 score. When compared with other competitive neural network architectures our approach demonstrates a superior tracking performance.
... We use the term trajectory reconstruction for estimating the AIS positions and connecting them as trajectories [6]. The existing works of trajectory reconstruction include linear interpolation, curvilinear interpolation [7], and its improvements [8,9], and Recurrent Neural Networks (RNNs) [10]. Some of these methods employ physical models of movement information such as speeds, directions, and time, and typically use the speed over ground and course over ground, and others assume a distribution of vessel trajectories and train it from historical records [11,12]. ...
A trajectory is a sequence of observations in time and space, for examples, the path formed by maritime vessels, orbital debris, or aircraft. It is important to track and reconstruct vessel trajectories using the Automated Identification System (AIS) data in real-world applications for maritime navigation safety. In this project, we use the National Science Foundation (NSF)'s Algorithms for Threat Detection program (ATD) 2019 Challenge AIS data to develop novel trajectory reconstruction method. Given a sequence of N unlabeled timestamped observations Χ={x 1 ,x 2 ,...,x N } , the goal is to track trajectories by clustering the AIS points with predicted positions using the information from the true trajectories Χ . It is a natural way to connect the observed point x î with the closest point that is estimated by using the location, time, speed, and angle information from a set of the points under consideration x i ∀ i ∈ {1, 2, …, N }. The introduced method is an unsupervised clustering-based method that does not train a supervised model which may incur a significant computational cost, so it leads to a real-time, reliable, and accurate trajectory reconstruction method. Our experimental results show that the proposed method successfully clusters vessel trajectories.
... Besides, some researches were also proposed to tackle the occlusion problem in visual object tracking (Pan et al., 2008;Jiang et al., 2018;Cui et al., 2021). However, in the field of maritime surveillance, the tracking targets of maritime traffic usually are vessels (Chen et al., 2020;Perera et al., 2012). The occlusion between vessels often takes a long time with large occlusion areas, even completely occluded. ...
To guarantee vessel traffic safety in inland waterways, the automatic identification system (AIS) and shore-based cameras have been widely adopted to monitor moving vessels. The AIS data could provide the unique maritime mobile service identity (MMSI), position coordinates (i.e., latitude and longitude), course over ground, and speed over ground for the vessels of interest. In contrast, the cameras could directly display the visual appearance of vessels but fail to accurately grasp the vessels' identity information and motion parameters. In this paper, we propose to improve the maritime traffic surveillance in inland waterways using the robust fusion of AIS and visual data. It is able to obtain more accurate vessel tracking results and kinematic characteristics. In particular, to robustly track the visual vessels under complex scenarios, we first propose an anti-occlusion vessel tracking method based on the simple online and real-time tracking with a deep association metric (DeepSORT) method. We then preprocess and predict the vessel positions to obtain synchronous AIS and visual data. Before the implementation of AIS and visual data fusion, the AIS position coordinates in the geodetic coordinate system will be projected into the image coordinate system via the coordinate transformation. A multi-feature similarity measurement-based Hungarian algorithm is finally proposed to robustly and accurately fuse the AIS and visual data in the image coordinate system. For the sake of repeating fusion experiments, we have also presented a new multi-sensor dataset containing AIS data and shore-based camera imagery. The quantitative and qualitative experiments show that our fusion method is capable of improving the maritime traffic surveillance in inland waterways. It can overcome the vessel occlusion problem and fully utilizes the advantages of multi-source data to promote the maritime surveillance, resulting in enhanced vessel traffic safety and efficiency. In this work, the presented multi-sensor dataset and source code are available at https://github.com/QuJX/AIS-Visual-Fusion.
... The representation of spatial data generated in various fields such as maritime traffic [88] and people movement analysis [11] is the first fundamental step in analyzing this data. Most of the analysis works represent this data as spatial trajectories, i.e. sequences of locations ordered by their timestamp. ...
The mission of the National Library of France (BnF) is to collect, preserve, enrich and make available the national documentary heritage. Its collections comprise nearly forty million documents.One of the BnF's missions is to maintain the documents of its collections in good condition in order to ensure their availability to readers.The definition of a conservation/restoration policy by the experts requires the identification of the documents in poor condition; to this end, the physical state of the documents must be monitored regularly to identify those requiring urgent interventions. But this time-consuming task is impossible in practice due to the large volume of documents.The objective of our work is to provide a support to the experts in the definition of their conservation/restoration policies and to provide a decision support system allowing the characterization of the physical state of documents by the integration and analysis of the data available in the databases of the various departments of the BnF.Considering that each document is described by a conservation/restoration history, which includes all the information likely to have an impact on its physical state, the main questions we are faced with are, on the one hand, the representation of these histories and their comparison taking into account their terminological heterogeneity, and on the other hand, the definition of an analysis process of these histories enabling to characterize the state of the documents and to predict it.Our work aims to propose some contributions towards a decision support system for conservation/restoration experts at the BnF. We have proposed a representation of conservation/restoration histories as semantic trajectories, and we have proposed appropriate similarity measures to resolve the terminological heterogeneity of the data using an external knowledge base developed in collaboration with experts. We also have defined an analysis process based on a clustering algorithm to predict the documents' physical state. Finally, we have proposed a novel concept weighting approach that allows to define the importance of the concepts considering a specific analysis task.
... Also, some researchers integrated anomaly detection methods into ship monitoring systems, which helped to detect anomalies in ship behavior in real-world scenarios. Perera et al. (2012) designed an artificial neural network (ANN)-based ship target detection and tracking model and an extended Kalman filter (EKF)-based ship state estimation and trajectory prediction model, and integrated the ANN and EKF into a ship traffic monitoring system to enhance safety and security services during ship navigation. ...
Maritime traffic safety influences the development of world economies. A major aspect to enhance maritime traffic safety is the effective detection of abnormal ship behavior (DASB) which recently have been widely made based on data-driven methods using multisource heterogeneous data. In order to provide an overview of the state-of-the-art of research, this study presents a review of DASB. First, the categories of abnormal ship behavior and data sources of DASB are concretely introduced, and the process of data-driven DASB combined with expert knowledge are described. Second, we conduct systematic disciplinary knowledge maps in the field of DASB, which identify evolution, hotspots and emerging trends. In this manner, data-driven methods for DASB were categorized into six types, including multisource data fusion approaches, statistical analysis approaches, traditional intelligent algorithmic approaches, deep learning approaches, knowledge-based and data-driven integrating approaches and computing power, and provide an overview of them. Then, we propose an integrated framework for DASB. Finally, we discuss the challenges in terms of three technical aspects (data, algorithm, and computing power) and outline possible paths of investigation for DASB to improve intelligent maritime surveillance.
... Datadriven approaches try to overcome this challenge by learning from historical traffic patterns and thus implicitly extracting representative behavior. Here, works with machine learning methods based on the extended Kalman filter [10], random forest [11], support vector machines [12] or particle filter [13] have been proposed. ...
We suggest a data-driven approach to predict vessel trajectories by mimicking the underlying policy of human captains. Decisions made by those experts are recorded by the automatic identification system (AIS) signals and can be fused with additional non-kinematic factors like destination, weather condition, current tide level or ship size to get a more accurate snapshot of the situation that led to chosen maneuvers. In this work, we explore the usage of a method meant for optimal control, namely Behavioral Cloning, in a forecasting problem, in order to generate end-to-end vessel trajectories purely based on a given initial state. The training and test datasets consist of trajectories from the coast of Bremerhaven, having more than one thousand unique ships and different motion clusters. These are processed by a single deep-learning model, showing promising results in terms of accuracy and providing a research avenue for a near real-time application where vessel trajectories are to be forecast from a given snapshot of the situation - not from the costly history of all the vessels present.
... With the rapid development of location sensing technologies, extensive trajectory datasets are available for recording movements of objects or events (Yu, 2019). Detecting anomalies for such trajectory data is critical to the understanding of movement patterns and behaviors, and thus has wide applications related to driver assistance systems (Kong et al., 2017;Wu et al., 2017), traffic monitor (Perera et al., 2012) and risk management (Kuang et al., 2015). ...
Traditional trajectory anomaly detection aims to find abnormal trajectory points or sequences using data mining techniques. As a comparison, we focus on the evaluation of the anomalies of driving habits for different drivers based on their trajectory data. This is particularly important for the application of adjusting the amount of insurance in accordance with the driving behaviors. Instead of customizing rules for modeling various driving behaviors, we propose an end-to-end deep learning framework for driving trajectory anomaly detection, called STDTB-AD. Specifically, taking into account the fact that movement is spatial–temporal dependent, the study first partitions the whole road network into a series of spatial–temporal units, which have homogeneous properties of traffic flows. Then, the motion parameters (i.e., acceleration, speed, and direction) of driving trajectories falling within each spatio-temporal unit are calculated for representing context-aware features of drivers. This method is able to detect the deviation of movement from the normal traffic state on the spatial–temporal units. Finally, a variational autoencoder is utilized to quantify the abnormity degree of each driver according to the reconstruction probability of driving feature vector. Evaluations based on taxi trajectory data show that our model can consider both the spatial and temporal contexts for detecting driving behavior anomalies and achieve higher detection accuracy than traditional models based on either spatial or temporal context.
... In the simplest models, conventional interpolation methods such as linear interpolation and curved interpolation are used. More sophisticated models construct the ship's kinematic equations and assimilate AIS observations using extended Kalman filters [13] and particle filters. Among the model-based predictors, the simplest and probably the most popular is the near-constant velocity (NCV) linear model [14]. ...
Data-driven technologies and automated identification systems (AISs) provide unprecedented opportunities for maritime surveillance. As part of enhancing maritime situational awareness and safety, in this paper, we address the issue of predicting a ship’s future trajectory using historical AIS observations. The objective is to use past data in the training phase to learn the predictive distribution of marine traffic patterns and then use that information to forecast future trajectories. To achieve this, we investigate an encoder–decoder architecture-based sequence-to-sequence prediction model and CNN model. This architecture includes a long short-term memory (LSTM) RNN that encodes sequential AIS data from the past and generates future trajectory samples. The effectiveness of sequence-to-sequence neural networks (RNNs) for forecasting future vessel trajectories is demonstrated through an experimental assessment using an AIS dataset.
... protocol. The existing work in maritime traffic management and surveillance have focused on prediction of destinations, trajectories, and anomaly detection in an observed ship irrespective of the nearby vessel information (Perera, Oliveira, and Guedes Soares 2012;Filipiak et al. 2018). Other related works model vessel behavior from historical AIS information using probabilistic models, e.g. ...
Maritime surveillance is essential to avoid illegal activities and for environmental protection. However, the unlabeled, noisy, irregular time-series data and the large area to be covered make it challenging to detect illegal activities. Existing solutions focus only on trajectory reconstruction and probabilistic models that do ignore the context, such as the neighboring vessels. We propose a novel representation learning method that considers both temporal and spatial contexts learned in a self-supervised manner, using a selection of pretext tasks that do not require to be labeled manually. The underlying model encodes the representation of maritime vessel data compactly and effectively. This generic encoder can then be used as input for more complex tasks lacking labeled data.
... Trajectory prediction is a popular research topic in various engineering application domains including cars, pedestrians [9], ships [10], aircraft [11], etc. In this section, we will discuss the extensive motion prediction research for groundbased autonomous systems. ...
Behavior prediction plays an important role in integrated autonomous driving software solutions. In behavior prediction research, interactive behavior prediction is a less-explored area, compared to single-agent behavior prediction. Predicting the motion of interactive agents requires initiating novel mechanisms to capture the joint behaviors of the interactive pairs. In this work, we formulate the end-to-end joint prediction problem as a sequential learning process of marginal learning and joint learning of vehicle behaviors. We propose ProspectNet, a joint learning block that adopts the weighted attention score to model the mutual influence between interactive agent pairs. The joint learning block first weighs the multi-modal predicted candidate trajectories, then updates the ego-agent's embedding via cross attention. Furthermore, we broadcast the individual future predictions for each interactive agent into a pair-wise scoring module to select the top $K$ prediction pairs. We show that ProspectNet outperforms the Cartesian product of two marginal predictions, and achieves comparable performance on the Waymo Interactive Motion Prediction benchmarks.
... The volume, uncertainty, and sequential nature of the spatio-temporal data fostered research on AIS-based solutions driven to monitor [8,9] and increase safety [10][11][12], expanding our awareness about the marine traffic [13][14][15]. Along with similar premises, extensive benchmark of neural network architectures for detecting fishing activity from AIS data streams, in which our proposed network achieves about 87% of the F-score on trajectories of unseen vessels. ...
Automatic Identification System (AIS) messages are useful for tracking vessel activity across oceans worldwide using radio links and satellite transceivers. Such data play a significant role in tracking vessel activity and mapping mobility patterns such as those found during fishing activities. Accordingly, this paper proposes a geometric-driven semi-supervised approach for fishing activity detection from AIS data. Through the proposed methodology, it is shown how to explore the information included in the messages to extract features describing the geometry of the vessel route. To this end, we leverage the unsupervised nature of cluster analysis to label the trajectory geometry, highlighting changes in the vessel’s moving pattern, which tends to indicate fishing activity. The labels obtained by the proposed unsupervised approach are used to detect fishing activities, which we approach as a time-series classification task. We propose a solution using recurrent neural networks on AIS data streams with roughly 87% of the overall F-score on the whole trajectories of 50 different unseen fishing vessels. Such results are accompanied by a broad benchmark study assessing the performance of different Recurrent Neural Network (RNN) architectures. In conclusion, this work contributes by proposing a thorough process that includes data preparation, labeling, data modeling, and model validation. Therefore, we present a novel solution for mobility pattern detection that relies upon unfolding the geometry observed in the trajectory.
... Such general-scope abilities mean that AI can be leveraged in many, if not all, domains of our societies. For example, among other things, AI has been found to outperform radiologists in breast cancer detection before symptoms appear (McKinney et al., 2020), it designs floorplans for microchip development faster and better than human engineers (Kahng, 2021), it is more accurate and more reactive than human agents in traffic monitoring (Perera et al., 2012) (for an exhaustive review of the application domains of AI, see OECD, 2019). ...
Innovation management has attracted considerable attention from management scholars in all disciplines given the critical importance of innovation activities to all firms. While our knowledge of the innovation process has advanced, there is little debate that the tasks within the main stages of the process remain not only resource-intensive but also fraught with ambiguity. In the meantime, Artificial Intelligence (AI) has been rising as a powerful general-purpose technology that promises to lower uncertainty with more accurate predictions and to reduce the cost of laborious tasks. Here, we perform a critical review of the literature at the intersection of artificial intelligence and innovation management to present our thoughts on the ways AI can augment innovation managers in dealing with their tasks at each of the four stages of the process. We also present the articles in this special issue and explain their contribution to advancing our knowledge on how AI can act as an enabler for innovation.
... MarineTraffic 1 and VesselFinder 2 ). 30 The volume, uncertainty, and sequential nature of the spatio-temporal data fostered 31 research on AIS-based solutions driven to monitor [8,9] and increase safety [10-12], ex-32 panding our awareness about the marine traffic [13][14][15]. Along with similar premises, 33 other authors used the AIS data to understand and forecast vessels' trajectories [16,17] 34 or understand their environmental impact [18,19]. ...
Automatic Identification System (AIS) messages are useful for tracking vessel activity across oceans worldwide using radio links and satellite transceivers. Such data plays a significant role in tracking vessel activity and mapping mobility patterns such as those found in fishing. Accordingly, this paper proposes a geometric-driven semi-supervised approach for fishing activity detection from AIS data. Through the proposed methodology we show how to explore the information included in the messages to extract features describing the geometry of the vessel route. To this end, we leverage the unsupervised nature of cluster analysis to label the trajectory geometry highlighting the changes in the vessel's moving pattern which tends to indicate fishing activity. The labels obtained by the proposed unsupervised approach are used to detect fishing activities, which we approach as a time-series classification task. In this context, we propose a solution using recurrent neural networks on AIS data streams with roughly 87% of the overall $F$-score on the whole trajectories of 50 different unseen fishing vessels. Such results are accompanied by a broad benchmark study assessing the performance of different Recurrent Neural Network (RNN) architectures. In conclusion, this work contributes by proposing a thorough process that includes data preparation, labeling, data modeling, and model validation. Therefore, we present a novel solution for mobility pattern detection that relies upon unfolding the trajectory in time and observing their inherent geometry.
... In recent decades, as the shipping industry of China has grown by leaps and bounds, water transportation has been confronted with issues, such as an increase in ship traffic density, the frequency of water traffic accidents, and the increasing difficulty of maritime safety supervision, all of which pose obstacles to the sustainable growth of the shipping industry [1,2]. AIS data, whose primary information is spatiotemporal data consisting of ship location and time, provides ship trajectory data that can be used to analyze ship navigation behavior in real time, as well as provide critical supplementary information in the process of collision avoidance [3,4]. ...
According to the statistics of water transportation accidents, collision accidents are on the rise as the shipping industry has expanded by leaps and bounds, and the water transportation environment has become more complex, which can result in grave consequences, such as casualties, environmental destruction, and even massive financial losses. In view of this situation, high-precision and real-time ship trajectory prediction based on AIS data can serve as a crucial foundation for vessel traffic services and ship navigation to prevent collision accidents. Thus, this paper proposes a high-precision ship track prediction model based on a combination of a multi-head attention mechanism and bidirectional gate recurrent unit (MHA-BiGRU) to fully exploit the valuable information contained in massive AIS data and address the insufficiencies in existing trajectory prediction methods. The primary advantages of this model are that it allows for the retention of long-term ship track sequence information, filters and modifies ship track historical data for enhanced time series prediction, and models the potential association between historical and future ship trajectory status information with the current state via the bidirectional gate recurrent unit. Significantly, the introduction of a multi-head attention mechanism calculates the correlation between the characteristics of AIS data, actively learns cross-time synchronization between the hidden layers of ship track sequences, and assigns different weights to the result based on the input criterion, thereby enhancing the accuracy of forecasts. The comparative experimental results also verify that MHA-BiGRU outperforms the other ship track prediction models, demonstrating that it possesses the characteristics of ease of implementation, high precision, and high reliability.
... Although most maritime traffic is very regular, and thus model-based methods can be easily applied, in the presence of maneuvering behaviors of the ship such models will tend to lack the desired prediction accuracy. In such cases, nonlinear and data-driven methods including adaptive kernel density estimation [4], nonlinear filtering [5], [6], nearest-neighbor search methods [7], [8], and machine learning techniques [9], may provide more suitable solutions. Furthermore, the latest advances in deep learning-based predictive models and the combined availability of large volumes of AIS data allow for enhanced vessel trajectory prediction and maritime surveillance. ...
Recent deep learning methods for vessel trajectory prediction are able to learn complex maritime patterns from historical Automatic Identification System (AIS) data and accurately predict sequences of future vessel positions with a prediction horizon of several hours. However, in maritime surveillance applications, reliably quantifying the prediction uncertainty can be as important as obtaining high accuracy. This paper extends deep learning frameworks for trajectory prediction tasks by exploring how recurrent encoder-decoder neural networks can be tasked not only to predict but also to yield a corresponding prediction uncertainty via Bayesian modeling of epistemic and aleatoric uncertainties. We compare the prediction performance of two different models based on labeled or unlabeled input data to highlight how uncertainty quantification and accuracy can be improved by using, if available, additional information on the intention of the ship (e.g., its planned destination).
... Ship trajectory prediction and abnormal trajectory detection are the two major perspectives for investigating the motion patterns of ships. For ship motion prediction, different methodologies like the Gaussian process model and the BP neural network algorithm have been proposed in previous studies (Rhodes et al., 2007;Zhou and Shi, 2010;Perera et al., 2012;Xu et al., 2012;Zhang et al., 2018;Rong et al., 2019). An accurate ship motion prediction can help maritime authorities to predict the possible activities of target ships. ...
Ship anomaly detection is a vital aspect for monitoring navigational safety in specific water areas. Considering the effect of water channel boundaries, we propose the detection of an abnormal ship trajectory based on the complex polygon (DATCP) method to detect ship anomalies in this study. With the automatic identification systems (AIS) data from the Yangtze River estuary, a case study is created to verify the effectiveness of the proposed DATCP method. The case study results reveal that the proposed DATCP method can provide higher detection accuracy than the conventional A* algorithm. The feature analysis results indicate that ship anomalies are significantly influenced by ship type, time period, weather conditions and ship traffic characteristics.
Detecting waypoints where vessels change their behavior (i.e., maneuvers, speed changes, etc.) is essential for optimizing vessel trajectories to increase the efficiency and safety of sailing. However, accurately detecting waypoints is challenging due to potential AIS data quality issues (i.e., missing or inaccurate messages). In this paper, we propose a five-step learning approach (SafeWay) to estimate waypoints on a given AIS trajectory. First, we interpolate trajectories to tackle AIS data quality issues. Then, we annotate historical trajectories by using an existing waypoint library that contains historical waypoints. As the historical waypoints are passage plans manually created by port operators considering sailing conditions at that time, they are not specific to other historical trajectories between the same ports. We, therefore, use a similarity metric to determine overlapping segments of historical trajectories with the historical waypoints from the waypoint library. Then, we build a transformer model to capture vessel movement patterns based on speed- and location-related features. We do not process location features directly to avoid learning location-specific context, but take into account tailored delta features. We test our approach on a real-world AIS dataset collected from the Norwegian Sea between Å lesund and Måløy and show its effectiveness in terms of a harmonic mean of purity and coverage, mean absolute error and detection rate on the task of detecting trajectory waypoints compared to a state-of-the-art approach. We also show the effectiveness of the trained model on the trajectories obtained from two other regions, the North Sea (London and Rotterdam) and the North Atlantic Ocean (Setubal and Gibraltar), on which the model has not been trained. The experiments indicate that our interpolation-enabled transformer design provides improvements in the safety of the estimated waypoints.
Automatic identification system (AIS) is a vessel-based system for the automatic broadcast and reception of vessel information, and it also supports data for trajectory prediction. Since the vessel’s sailing route is flexible and changeable and the AIS broadcast is unconfirmed, the trajectory varies greatly and the original AIS data contains some noisy trajectory, which leads to low prediction accuracy and stability. Therefore, to solve the above problem, this paper proposes a trajectory prediction method based on bidirectional gate recurrent unit (BiGRU) and trajectory direction vector (TDV) with attention mechanism. This paper firstly proposes a TDV to associate latitude and longitude with the course and speed. Then the paper proposes an attention mechanism to self-adaptively update weight to the TDV in different stages to eliminate unreasonable predicted trajectory points. Finally, this paper combines the TDV attention mechanism and the BiGRU network to train a vessel trajectory prediction model.
The trajectory prediction of maritime targets is an urgent problem to be solved in order to effectively manage and control the situation in the sea area. The UAV has the advantages of strong real-time performance, high resolution and relatively stable viewing angle. Trajectory prediction of targets at sea. In this paper, a method of ship track prediction based on sliding window based bidirectional long short-term memory network (Bi-LSTM) is proposed using ship track data. Considering the high feature dimension of ship position information, the LSTM network theory is used to construct a ship track prediction model based on the sliding window BLSTM network, and the spatiotemporal features of the ship are extracted, and the position, speed and heading output of the network are used to predict the ship track. The simulation results show that, compared with the prediction results of LSTM and RNN, the prediction accuracy and generalization ability of the sliding window BLSTM network are better in the direct flight, steering and maneuvering scenarios.
A growing interest in developing autonomous surface vehicles (ASVs) has been witnessed during the past two decades, including COLREGs-compliant navigation to ensure safe autonomy of ASVs operating in complex waterways. This paper reviews the recent progress in COLREGs-compliant navigation of ASVs from traditional to learning-based approaches. It features a holistic viewpoint of ASV safe navigation, namely from collision detection to decision making and then to path replanning. The existing methods in all these three stages are classified according to various criteria. An in-time overview of the recently-developed learning-based methods in motion prediction and path replanning is provided, with a discussion on ASV navigation scenarios and tasks where learning-based methods may be needed. Finally, more general challenges and future directions of ASV navigation are highlighted.
In this paper, finite-time trajectory tracking problem of the marine vessels (MVs) disturbed by highly coupled nonlinearities and unknown ocean current is solved by developing a barrier function-based adaptive fast non-singular terminal sliding mode control (BFAFNTSMC) method. Salient features are as follows: (1) The defined fast non-singular terminal sliding mode manifold (FNTSM) which introduces the finite-time ocean current observer (OCO) is proposed to guarantee the finite-time sliding motion for mismatched disturbance attenuation. (2) By devising a novel log-type barrier function adaptive sliding mode reaching law (BFARL) based on single switching logic, the synovial variable will converge to and maintain in a predefined neighborhood of zero in a finite time, without overestimating the control gain. (3) Finite-time convergence of trajectory tracking errors to the predefined neighborhood of zero can be guaranteed, independently of the boundaries of the lumped nonlinearities. The proposed method exhibits the satisfied properties of tracking performance as well as chattering alleviation. Simulation results and comparisons are further presented to illustrate remarkable effectiveness of the proposed approach.
Intelligent maritime transportation is one of the most promising enabling technologies for promoting trade efficiency and releasing the physical labor force. The trajectory prediction method is the foundation to guarantee collision avoidance and route optimization for ship transportation. This article proposes a bidirectional data-driven trajectory prediction method based on Automatic Identification System (AIS) spatio-temporal data to improve the accuracy of ship trajectory prediction and reduce the risk of accidents. Our study constructs an encoder-decoder network driven by a forward and reverse comprehensive historical trajectory and then fuses the characteristics of the sub-network to predict the ship trajectory. The AIS historical trajectory data of US West Coast ships are employed to investigate the feasibility of the proposed method. Compared with the current methods, the proposed approach lessens the prediction error by studying the comprehensive historical trajectory, and 60.28% has reduced the average prediction error. The ocean and port trajectory data are analyzed in maritime transportation before and after COVID-19. The prediction error in the port area is reduced by 95.17% than the data before the epidemic. Our work helps the prediction of maritime ship trajectory, provides valuable services for maritime safety, and performs detailed insights for the analysis of trade conditions in different sea areas before and after the epidemic.
Maritime traffic situational awareness is fundamental to the safety of maritime transportation. The state-of-the-art research primarily attaches importance to collision risk estimation and evaluation between/among ships but encounters the challenges of capturing the high-risk traffic clusters in complex waters. This paper develops a systematic traffic clustering approach to enhance traffic pattern interpretability and proactively discover high-risk multi-ship encounter scenarios, in which both the conflict connectivity and spatial compactness of encounter ships are considered. Specifically, a novel hybrid clustering approach that integrates a composite distance measure, a constrained Shared Nearest Neighbour clustering, and a fine-tuning strategy is developed to segment maritime traffic into multiple conflict-connected and spatially compact clusters. Meanwhile, a hierarchical bi-objective optimization algorithm is introduced to search for optimal clustering solutions. Through maritime traffic data obtained from the Ningbo-Zhoushan Port, a thorough methodology performance evaluation is carried out through application demonstration and validation. Experiment results reveal that the new approach: 1) can effectively capture the high-risk/density traffic clusters; 2) is robust with respect to various traffic scenarios; and 3) can be extended to assist in collision risk management. It therefore offers new insights into enhancing maritime traffic surveillance capabilities and eases the design of risk management strategy.
The marine vessel Automatic Identification System (AIS) broadcast is a system with no-response mechanism, so there are missing data from the received AIS vessel trajectory. Currently, the mainstream vessel trajectory curve fitting method is achieved by means of deep learning algorithm cyclic training, however, the quality of the original data has a large impact on the trajectory fitting results, and the current method does not consider the interference of missing trajectory points on the fitted trajectory curve. Therefore, based on the data features of AIS, this paper proposes a lightweight deep learning method: Forward Backward Bidirectional Gated Recurrent Unit (FB-BiGRU). The method in this paper consists of a forward Gated Recurrent Unit (GRU) network and a backward Bidirectional Gated Recurrent Unit (BiGRU) network, reducing the range of the trajectory to be fitted by scaling in two directions simultaneously, thus gradually realizing the trajectory fitting function. The characteristics of our method are that training the trajectory data by bi-directional scaling can maximize the linear features of the trajectory curves to enhance the accuracy and efficiency of linear regression with no additional computing resources. The trajectory fitting performance of this method is verified on actual Denmark trajectory datasets, we prove that the accuracy of our proposed method in fitting short-term trajectories has increased by 49.16% and 29.89% on average compared with the (Long Short-Term Memory) LSTM and BiGRU. Furthermore, the average fitting accuracy of our method is 96 m, and the minimum fitting error is 64 m.
In this paper, we present an automated process for detecting the anomaly in Automatic Identification System (AIS) data. Machine learning approaches have been employed to automatically detect anomalies in the AIS data. The opensource AIS data is been used to evaluate the performance of the proposed approach. Supervised machine learning approaches like K Nearest Neighbor, Random Forest, Support Vector Machine classifier is employed to predict the anomalies in the AIS data. The AIS data does not contain the ground truth labels and supervised learning algorithms need labelling data, to address this issue, we employed an unsupervised approach to label the data based on the prior information and characteristics of the AIS data. The labelled data is then used to train the supervised machine learning models. The proposed approach with support vector machine classifier has classified the AIS data into normal and anomaly with an accuracy of 96.5%.
Autonomous ships are expected to improve the level of safety and efficiency in future maritime navigation. Such vessels need perception for two purposes: to perform autonomous situational awareness and to monitor the integrity of the sensor system itself. In order to meet these needs, the perception system must fuse data from novel and traditional perception sensors using Artificial Intelligence (AI) techniques. This article overviews the recognized operational requirements that are imposed on regular and autonomous seafaring vessels, and then proceeds to consider suitable sensors and relevant AI techniques for an operational sensor system. The integration of four sensors families is considered: sensors for precise absolute positioning (Global Navigation Satellite System (GNSS) receivers and Inertial Measurement Unit (IMU)), visual sensors (monocular and stereo cameras), audio sensors (microphones), and sensors for remote-sensing (RADAR and LiDAR). Additionally, sources of auxiliary data, such as Automatic Identification System (AIS) and external data archives are discussed. The perception tasks are related to well-defined problems, such as situational abnormality detection, vessel classification, and localization, that are solvable using AI techniques. Machine learning methods, such as deep learning and Gaussian processes, are identified to be especially relevant for these problems. The different sensors and AI techniques are characterized keeping in view the operational requirements, and some example state-of-the-art options are compared based on accuracy, complexity, required resources, compatibility and adaptability to maritime environment, and especially towards practical realization of autonomous systems.
Currently, the capabilities to capture, store and process logistics data, such as generated by the transport and handling of millions of maritime containers to distribute cargo worldwide, are available. A lot of these logistics events are already recorded and stored within some databases to keep track of operations. These data represent considerable value when analysed to diagnose bottlenecks and inefficiencies and guide better decisions in global supply chains. Since, amongst other things, the data is not readily available as information to the decision maker, this potential has not been reaped. In this paper, we focus on the question of how data can be transformed into meaningful information to the decision maker even when data is available to a limited extent. We explore the role of data-driven 4PL IT platforms, where users of the platform provide data that is incomplete and untimely, in producing valuable information for the stakeholders of their logistics ecosystem. We develop a mathematical model to obtain meaningful information from lower-quality data. We apply this in the context of container logistics of river vessels (barges) in a port environment. We introduce three sets of functions that capture movement, inventory, and productivity, to describe the logistics processes at hand and assess the state of a distribution network, often not recorded by the IT systems of operators in the distribution network. A Kalman filter approach is used to match movement and productivity information, to detect the state of the distribution network, and to predict its evolution in support of decision making about the allocation of containers to empty slots on barges.
The growing availability of maritime IoT traffic data and continuous expansion of the maritime traffic volume, serving as the driving fuel, propel the latest Artificial Intelligence (AI) studies in the maritime domain. Among the most recent advancements, vessel trajectory prediction is one of the most essential topics for assuring maritime transportation safety, intelligence, and efficiency. This paper presents an up-to-date review of existing approaches, including state-of-the-art deep learning, for vessel trajectory prediction. We provide a detailed explanation of data sources and methodologies used in the vessel trajectory prediction studies, highlight a discussion regarding the auxiliary techniques, complexity analysis, benchmarking, performance evaluation, and performance improvement for vessel trajectory prediction research, and finally summarize the current challenges and future research directions in this field.
Understanding and representing traffic patterns are key to detecting anomalous trajectories in the transportation domain. However, some trajectories can exhibit heterogeneous maneuvering characteristics despite confining to normal patterns. Thus, we propose a novel graph-based trajectory representation and association scheme for extraction and confederation of traffic movement patterns, such that data patterns and uncertainty can be learned by deep learning (DL) models. This paper proposes the usage of a recurrent neural network (RNN)-based evidential regression model, which can predict trajectory at future timesteps as well as estimate the data and model uncertainties associated, to detect anomalous maritime trajectories, such as unusual vessel maneuvering, using automatic identification system (AIS) data. Furthermore, we utilize evidential deep learning classifiers to detect unusual turns of vessels and the loss of transmitted signal
using predicted class probabilities with associated uncertainties.
Our experimental results suggest that the graphical representation
of traffic patterns improves the ability of the DL models, such
as evidential and Monte Carlo dropout, to learn the temporal-spatial
correlation of data and associated uncertainties. Using different datasets and experiments, we demonstrate that the estimated prediction uncertainty yields fundamental information for the detection of traffic anomalies in the maritime and, possibly
in other domains.
Considering the importance of timeliness in ship risk assessment, a high-resolution and long sequence multi-step trajectory prediction method is proposed. Through the multi-step prediction and uncertainty analysis of the ship trajectory for a long period of time, it is possible to make timely ship navigation risk assessment. First, the ship trajectory with high resolution is obtained by cubic spline interpolation. Then, Dynamic Time Warping (DTW) is used as the metric of distance, and a method called Laplacian Eigenmaps Self-Organizing Map (LE-SOM) is used to extract the features from original high-dimensional data with unequal intervals, so as to select the historical trajectories that can be used as a reference. Finally, the trajectory is generated for multi-step prediction. The proposed method not only predicts the position of the ship and its uncertainty from statistical perspective, but also investigates the relationship between trajectory curvature and the prediction error. The case study on a ferry ship in the Jiangsu section of the Yangtze River indicates the validity of the method.
This article investigates the trajectory tracking control of fully actuated unmanned surface vehicles (USVs) in the presence of lumped system uncertainty and input saturation. To address this problem, an output feedback scheme, which combines the saturated adaptive super twisting algorithm and non-singular fast terminal sliding mode (SASTA-NFTSM) technique, is proposed. First, the velocities and lumped system uncertainty including external disturbances and model uncertainties, are estimated by a finite-time extended state observer (FTESO), simultaneously in finite time. Second, to deal with the saturation constraints on actuators, a novel sliding mode reaching law scheme, namely SASTA, that consists of an anti-windup compensator and a super twisting algorithm based on switching logic adaptive gains is proposed. Finally, the trajectory tracking control of USVs is solved with the non-trivial combination of SASTA and NFTSM. The chattering phenomenon can be effectively attenuated by virtue of the continuous control laws compared with the traditional NFTSM control (NFTSMC) approaches. The tracking errors are guaranteed to converge to zero in finite time. Comparative simulation results demonstrate the effectiveness and advantage of the proposed scheme.
The increasing attention and rapid development of intelligent ship technologies makes it possible to real applications to merchant shippings. The mixture of conventional and intelligent ships will be the most likely new scenario in the future. Multi-ship collision avoidance decision-making and the cooperation between them are the keys to ensure safety. To make decisions that both meet the requirements of safety and the principles of International Regulations for Preventing Collision at Sea (COLREGs), collision avoidance decision-making models for conventional and intelligent ships are formulated, respectively. The former is based on the COLREGs action clauses and human thinking patterns, whereas the latter is formulated through quantitative analysis of collision risk, rule compliance, ship yaw angle and drift distance. A collision-avoidance intention notification mechanism is formed, and the sequential decision is performed that enables real-time decision update. The simulation results show that the closest distances between ships in all scenarios are not less than the predefined safe distance, and ships comply with COLREGs. In the mixed scenarios, the intention notification mechanism can adapt to the impact of COLREGs violation by one ship, so as to reduce the operation complexity and course change frequency.
In this paper, a novel prescribed performance and full state constraints dynamic surface control (PP&FSC-DSC) scheme is proposed for trajectory tracking of unmanned surface vessel (USV) with model uncertainties, unknown nonlinear disturbances and full state and input constraints. Firstly, the restricted system with inequality constraints is transformed into the system with equality constraints by the modified asymmetric barrier Lyapunov function (MABLF) and then combined with the prescribed performance function, the vessel position tracking error meets the prescribed performance. Secondly, owing to use the dynamic surface technique and introduce an auxiliary dynamic system (ADS), the double advantages are established, i.e., the negative effects of the differential operation for the intermediate control law and input saturation of the intermediate control law are avoided. Thirdly, the model uncertainties are approximated by the neural network (NN), and the NN approximation error and unknown nonlinear disturbances is adaptively estimated. Finally, according to the Lyapunov stability theory, all signals are semi-globally uniformly ultimately bounded (SGUUB). The trajectory can be maintained with prescribed tracking performance and the full state can be constrained. The effectiveness of the proposed control protocol is validated by the simulation results.
We present a promising system to simultaneously detect and track multiple pedestrians in the outside scene using laser and vision. The useful information of laser and vision is automatically extracted and combined in a Bayesian formulation. In order to compute MAP estimation, an effective Probabilistic Detection-based Particle Filter (PD-PF) has been proposed. Experiments and evaluations demonstrate that not only can our system perform robustly in real environments, but also obtain better approximation of MAP than previous methods in most complex situations.
The detection and classification of moving objects is a key task in many control systems. This paper presents a solution for this problem based on a laser range scanner. The method can be summarized as follows. A set of 3D data points on the object sur- face is obtained using the scanner. A small number of features is then computed to represent the object boundary. In fact, three alternative features are con- sidered. Classification algorithms are then designed and evaluated using real data. The experimental tests show that the proposed techniques allow a robust classification of moving objects based on range in- formation.
In the presence of partial beam blockage (PBB), weather radar measurements can experience significant bias that directly compromises the accuracy of the hydrologic applications. Techniques for the calibration of the radar reflectivity factor Z and differential reflectivity ZDR, measured with dual-polarization weather radars in the presence of partial beam obstruction, are examined in this paper.
The proposed ZDR calibration technique utilizes radar measurements of ZDR in light rain and dry aggregated snow at unblocked and blocked elevations. This calibration technique was tested for the National Severe Storms Laboratory’s (NSSL’s) Cimarron radar that suffers from PBB, and a polarimetric prototype of the Weather Surveillance Radar-1988 Doppler (WSR-88D) that does not experience PBB. Results indicate that the ZDR bias that is associated with PBB can be calibrated with an accuracy of 0.2–0.3 dB, provided that the dataset is sufficiently large.
Calibration of Z in the presence of PBB is based on the idea of self-consistency among Z, ZDR, and the specific differential phase KDP in rain. The self-consistency calibration of Z from the Cimarron radar is performed following an area–time integral method. Integration is partitioned into small azimuthal sectors to assess the azimuthal modulation of the Z bias. The suggested technique is validated by direct comparisons of reflectivity factors that are measured by the Cimarron radar and the unobstructed operational WSR-88D radar. It is shown that the azimuthal modulation of Z that is caused by PBB is well captured, and the accuracy of the Z calibration is within 2–3 dB.
All over the world traffic injuries and fatality rates are increasing every year. The combination of negligent and imprudent drivers, adverse road and weather conditions produces tragic results with dramatic loss of life. In this scenario, the use of mobile robotics technology onboard vehicles could reduce casualties. Obstacle motion tracking is an essential ability for car-like mobile robots. However, this task is not trivial in urban environments where a great quantity and variety of obstacles may induce the vehicle to take erroneous decisions. Unfortunately, obstacles close to its sensors frequently cause blind zones behind them where other obstacles could be hidden. In this situation, the robot may lose vital information about these obstructed obstacles that can provoke collisions. In order to overcome this problem, an obstacle motion tracking module based only on 2D laser scan data was developed. Its main parts consist of obstacle detection, obstacle classification, and obstacle tracking algorithms. A motion detection module using scan matching was developed aiming to improve the data quality for navigation purposes; a probabilistic grid representation of the environment was also implemented. The research was initially conducted using a MatLab simulator that reproduces a simple 2D urban-like environment. Then the algorithms were validated using data samplings in real urban environments. On average, the results proved the usefulness of considering obstacle paths and velocities while navigating at reasonable computational costs. This, undoubtedly, will allow future controllers to obtain a better performance in highly dynamic environments.
The neural extended Kalman filter is a technique that learns unmodelled dynamics while performing state estimation. This coupled system performs the state estimation of the plant while estimating a function approximation of the difference between the system model and the dynamics of the true plant. At each sample step, this approximation is added to the existing model improving the state estimate. This neural estimator is applied to a two-dimensional intercept problem as a target tracker providing the control reference signal. Comparisons between different prediction times used in the control are provided for both the neural tracker and a baseline tracker. Copyright © 2005 IFAC
Maneuvering vessel detection and tracking in cooperation with vessel state estimation and navigational trajectory prediction are important tasks for the Vessel Traffic Monitoring and Information Systems (VTMIS) to improve maritime safety and security in ocean navigation. In this study, collaborated and constrained Neural-EKF algorithm is proposed for the above purpose. The proposed methodology consists of two main units: an Artificial Neural Network based Vessel Detection and Tracking Unit and an Extended Kalman Filter based State Estimation and Trajectory Prediction Unit. Finally, the proposed algorithm, is implemented on the MATLAB software platform, and successfully illustrate the results attainable in respect to vessel detection and tracking, vessel state estimation and navigational trajectory prediction in ocean navigation is also presented in this study.
To improve safety and security issues, maneuvering target detection and tracking are important facilities for navigation systems. Therefore, conventional navigation systems are equipped with Radar-based systems for the same purpose. However, Radar systems suffer some practical problems that are associated with the targets in close quarter navigation. Furthermore, Radar singles attenuate with distance, weather (ie. rain) and sea conditions, where the target tacking performances are degraded. Therefore, a Laser Measurement System (LMS) is proposed in this study to overcome the problems faced by the conventional Radar systems at close quarter navigation as well as bad weather and environmental conditions. Furthermore, capabilities of a LMS to measure accurate distance in close proximity as well as to observe the shape and size of the target are illustrated. In this study, each target is approximated by a cluster of data points rather than a single point target that is the main contribution in this paper. The adaptive Neural Network approach is proposed as a method of tracking maneuvering targets that are represented by clusters of data points. Successful simulation and experimental results of target detection and tracking that are tested on a experimental platform, SICK© LMS, are also presented in this paper.
This paper focuses on a study of the sequential action execution module for a collision avoidance system in ocean navigation. The overall decision-action process of collision avoidance consists on a Fuzzy logic based parallel decision making module and those decisions are formulated into collision avoidance actions by a Bayesian network based sequential action execution module. The presented collision avoidance system is capable of making multiple sequential actions to avoid complicated collision situations involving multiple vessels in ocean navigation while still respecting the COLREGs rules and regulations.
In this paper, an unscented Kalman filter (UKF) for curvilinear motions in an interacting multiple model (IMM) algorithm to track a maneuvering vehicle on a road is investigated. Driving patterns of vehicles on a road are modeled as stochastic hybrid systems. In order to track the maneuvering vehicles, two kinematic models are derived: A constant velocity model for linear motions and a constant-speed turn model for curvilinear motions. For the constant-speed turn model, an UKF is used because of the drawbacks of the extended Kalman filter in nonlinear systems. The suggested algorithm reduces the root mean squares error for linear motions and rapidly detects possible turning motions.
A feature detection system has been developed for real-time identification of lines, circles and people legs from laser range data. A new method suitable for arc/circle detection is proposed: the Inscribed Angle Variance (IAV). Lines are detected using a recursive line fitting method. The people leg detection is based on geometrical relations. The system was implemented as a plugin driver in Player, a mobile robot server. Real results are presented to verify the effectiveness of the proposed algorithms in indoor environment with moving objects.
We present GATMO (Generalized Approach to Tracking Movable Objects), a system for localization and mapping that incorporates the dynamic nature of the environment while maintaining semantic labels. Objects in the environment are broken down into multiple mobility levels, from static (walls) to highly mobile (people), by maintaining a history of object movement. Object classification is accomplished through a multi-layer, multi-hypothesis approach that does not rely on any static features such as shape or size. Maps are stored in an efficient manner that incorporates a history of previous orientations of each object. GATMO is initialized with a static map; it subsequently changes the map over time as objects in the map change position.
A new maritime navigation system based on a laser range-finder scanner for obstacle avoidance and precise maneuvering operations is described in this paper. The main novelty of this work is the adaptation and implementation of known technology for laser range finding and algorithms for target tracking into a system that operates in real time and has been tested in different natural sea and inland navigation scenarios. The principal components of this system, namely, 1) the laser range finder, 2) the scanning unit, and 3) the data processing and displaying unit, are described in detail. Ladar images are dense horizontally and sparse vertically as a compromise between capturing relevant features and quick frame formation. Images are processed for range outlier removal, and significant observable patterns are extracted. This multiple-target tracking problem is tackled using robust Kalman filtering techniques for continuous tracking of each detected observation. We minimize unreliable track initializations and preserve tracks from deletion during temporal misobservations. The evaluation in open-sea and inland waterways gave good results, making the system valid for precise maneuvering, fluent navigation, and accident mitigation. Objects of interest, from boats to ships, are detected and robustly tracked; pier and lock chamber sketches are reliable; bridge height estimation is precise; and narrow waterways (river banks and bridge columns) are correctly detected. The prototype developed can be considered to be a very valuable complementary device to traditional radar-based techniques that are not totally valid for accurate short-range exploration, improving efficiency and safety in ship operations.
A visual target tracking identification by employing using a ldquowinner-takes-allrdquo artificial neural network is proposed in this paper. In this approach a modified Kohonen neural network is the mechanism used both to determine the position as to represent the target trajectory given a sequence of images. Some of the advantages employing this technique is that the initial condition are supplied randomly and that the performance of the algorithm is independent of the initial condition as well as of the number of them. Besides, this algorithm converge for the center of mass of the target. This methodology is useful in remote and local systems when information is given by images be it related to aerospace applications, robotics, radar systems, or industrial applications. The proposed algorithm is here used in the identification of airplane trajectory by using digital images.
— For robots operating in real-world environments, the ability to deal with dynamic entities such as humans, animals, vehicles, or other robots is of fundamental importance. The variability of dynamic objects, however, is large in general, which makes it hard to manually design suitable models for their appearance and dynamics. In this paper, we present an unsupervised learning approach to this model-building problem. We describe an exemplar-based model for representing the time-varying appearance of objects in planar laser scans as well as a clustering procedure that builds a set of object classes from given training sequences. Extensive experiments in real environments demonstrate that our system is able to autonomously learn useful models for, e.g., pedestrians, skaters, or cyclists without being provided with external class information. I.
We present an approach to laser-based people tracking using a multi-hypothesis tracker that detects and tracks legs separately with Kalman filters, constant velocity motion models, and a multi-hypothesis data association strategy. People are defined as high-level tracks consisting of two legs that are found with little model knowledge. We extend the data association so that it explicitly handles track occlusions in addition to detections and deletions. Additionally, we adapt the corresponding probabilities in a situation-dependent fashion so as to reflect the fact that legs frequently occlude each other. Experimental results carried out with a mobile robot illustrate that our approach can robustly and efficiently track multiple people even in situations of high levels of occlusion.
The purpose of this report is to introduce an adaptive estimation and parameter identification scheme which we shall call Multiple Model Estimation algorithm (MMEA). The MMEA consists of a bank of Kalman filters with each matched to a possible parameter vector. The state estimates generated by these Kalman filters are then combined using a weighted sum with the a posteriori hypothesis probabilities as weighhting factors. If one of the selected parameter vectors coincides with the true parameter vector, this algorithm givens the minimum variance state and parameter estimates. Algorithms for filtering, smoothing, and prediction are derived for linear and nonlinear systems. They are described in a tutorial fashion with results stated explicitly so that they can be readily used for computer implementation. Approaches for the extension of MMEA to a more general class of adaptive estimation problems are outlined. several further research topics are also suggested.
A target tracking algorithm able to identify the position and to pursuit moving targets in video digital sequences is proposed in this paper. The proposed approach aims to track moving targets inside the vision field of a digital camera. The position and trajectory of the target are identified by using a neural network presenting competitive learning technique. The winning neuron is trained to approximate to the target and, then, pursuit it. A digital camera provides a sequence of images and the algorithm process those frames in real time tracking the moving target. The algorithm is performed both with black and white and multi-colored images to simulate real world situations. Results show the effectiveness of the proposed algorithm, since the neurons tracked the moving targets even if there is no pre-processing image analysis. Single and multiple moving targets are followed in real time.
In this paper we present a method of detection and tracking of moving objects (DATMO) using a laser range finder (LRF). The DATMO system is able to classify several kinds of objects and can be easily expanded to detect new ones. It is composed by three modules: scan segmentation; object classification using a suitable voting scheme of several object properties; and object tracking using a Kalman filter that takes the object type to increase the tracking performance into account. The goal is to design a collision avoidance algorithm to integrate a Cybercar vehicle, which uses the computed time-to-collision for each moving obstacle validated by the DATMO system.
The early recognition of potentially harmful traffic situations is an important goal of vision-based driver assistance systems. Pedestrians, in particular children, are highly endangered in inner city traffic. Within the DaimlerChrysler urban traffic assistance (UTA) project, we are using stereo vision and motion analysis in order to manage those situations. The flow/depth constraint combines both methods in an elegant way and leads to a robust and powerful detection scheme. A ball bouncing on the road often implies a child crossing the street. Since balls appear very small in the images of our cameras and can move considerably fast, a special algorithm has been developed to achieve maximum recognition reliability.
Classification is one of the most active research and application
areas of neural networks. The literature is vast and growing. This paper
summarizes some of the most important developments in neural network
classification research. Specifically, the issues of posterior
probability estimation, the link between neural and conventional
classifiers, learning and generalization tradeoff in classification, the
feature variable selection, as well as the effect of misclassification
costs are examined. Our purpose is to provide a synthesis of the
published research in this area and stimulate further research interests
and efforts in the identified topics
The detection of objects in high-resolution aerial imagery has
proven to be a difficult task. In the authors' application, the amount
of image clutter is extremely high. Under these conditions, detection
based on low-level image cues tends to perform poorly. Neural network
techniques have been proposed in object detection applications due to
proven robust performance characteristics. A neural network filter was
designed and trained to detect targets in thermal infrared images. The
feature extraction stage was eliminated and raw gray levels were
utilized as input to the network. Two fundamentally different approaches
were used to design the training sets. In the first approach, actual
image data were utilized for training. In the second case, a model-based
approach was adopted to design the training set vectors. The training
set consisted of object and background data. The neuron transfer
function was modified to improve network convergence and speed and the
backpropagation training algorithm was used to train the network. The
neural network filter was tested extensively on real image data.
Receiver operating characteristic (ROC) curves were determined in each
case. The detection and false alarm rates were excellent for the neural
network filters. Their overall performance was much superior to that of
the size-matched contrast-box filter, especially in the images with
higher amounts of visual clutter
A model of target motion in three-dimensional space that includes position derivatives up to the third order is developed. Compared with available models, which include terms at the most up to the second derivative, the model introduced in this work, called the jerk model, can more accurately describe agile target maneuvers which are likely to contain significant higher order derivatives. A compatible 4-state Kalman filter to perform tracking in conjunction with the jerk model is also presented, and an initialization procedure for the filter is provided. The improved performance of the jerk model over a lower order model is illustrated through numerical simulation.
Mobile robots and vehicles are increasingly used in dynamic environments populated by humans and other moving objects and vehicles. In this context, tracking of surrounding moving objects is important for obstacle avoidance and motion planning. In this paper we present a method for detection and tracking of multiple moving objects using particle filters to estimate the object states, and sample based joint probabilistic data association filters to perform the assignment between the features detected in the input sensor data and filters. Filters management operations are required for appropriate integration of the currently perceived features. A real-time architecture, developed to implement the tracking system, is briefly described. Experimental results obtained with a laser range scanner will be presented demonstrating the feasibility and effectiveness of the presented methods. 1)
A novel stochastic neural network is proposed. Unlike the traditional Boltzmann machine, the new model uses stochastic connections rather than stochastic activation functions. Each neuron has very simple functionality but all of its synapses are stochastic. It is shown that the stationary distribution of the network uniquely exists and it is approximately a Boltzmann-Gibbs distribution. It is also revealed there exists a strong relationship between the model and the Markov random field. New efficient techniques are developed to implement simulated annealing and Boltzmann learning. The model is successfully applied to a large-scale face recognition task in which face images are dynamically captured from a video source. Learning and recognizing processes are carried out in real time. The experimental results show the new model is not only feasible but also efficient.
A generalized, optimal filtering solution is presented for the target tracking problem. Applying optimal filtering theory to the target tracking problem, the tracking index, a generalized parameter proportional to the ratio of the position uncertainty due to the target maneuverability to that due to the sensor measurement, is found to have a fundamental role not only in the optimal steady-state solution of the stochastic regulation tracking problem, but also in the track initiation process. Using the tracking index parameter, an initializing and tracking procedure in recursive form, realizes the accuracy of the Kalman filter with an algorithm as simple as the well-known alpha minus beta filter or alpha minus beta minus gamma filter depending on the tracking order.
A fast target maneuver detecting and highly accurate tracking technique using a neural fuzzy network based on Kalman filter is proposed in this paper. In the automatic target tracking system, there exists an important and difficult problem: how to detect the target maneuvers and fast response to avoid misstracking? The traditional maneuver detection algorithms, such as variable dimension filter (VDF) and input estimation (IE) etc., are computation intensive and difficult to implement in real time. To solve this problem, neural network algorithms have been issued recently. However, the normal neural networks such as backpropagation networks usually produce the extra problems of low convergence speed and/or large network size. Furthermore, the way to decide the network structure is heuristic. To overcome these defects and to make use of neural learning ability, a developed standard Kalman filter with a self-constructing neural fuzzy inference network (KF-SONFIN) algorithm for target tracking is presented in this paper. By generating possible target trajectories including maneuver information to train the SONFIN, the trained SONFIN can detect when the maneuver occurred, the magnitude of maneuver values and when the maneuver disappeared. Without having to change the structure of Kalman filter nor modeling the maneuvering target, this new algorithm, SONFIN, can always find itself an economic network size with a fast learning process. Simulation results show that the KF-SONFIN is superior to the traditional IE and VDF methods in estimation accuracy.
This is the third part of a series of papers that provide a
comprehensive survey of the techniques for tracking maneuvering targets
without addressing the so-called measurement-origin uncertainty. Part I
and Part II deal with general target motion models and ballistic target
motion models, respectively. This part surveys measurement models,
including measurement model-based techniques, used in target tracking.
Models in Cartesian, sensor measurement, their mixed, and other
coordinates are covered. The stress is on more recent advances topics
that have received more attention recently are discussed in greater
details.
1. SUMMARY This paper introduces to the lecture series dedicated to the knowledge-based radar signal and data processing. Knowledge-based expert system (KBS) is in the realm of artificial intelligence. KBS consists of a knowledge base containing information specific to a problem domain and an inference engine that employs reasoning to yield decisions. KBS have been built: some are very complex with thousands rules while others, relatively simple, are designed to tackle very specialised tasks. This lecture series shows that KBS can be successfully applied to radar systems. This paper introduces the Reader to the world of radar and, specifically, to the topics tackled in the subsequent lectures of the series. The paper starts with an introduction (Section 2) to radar (radar evolution from the early days up today, taxonomy of radar and radar equation). Subsequently, Section 3 considers the schematic of a modern radar system. The phased-array radar is the theme of Section 4. Signal processing, one of the main building blocks of modern radar, is introduced in Section 5. The section also introduces to the various forms of adaptivity in time, space and space-time domains for natural and intentional interference mitigation. Data processing, mainly target tracking, (Section 6) is the other relevant building block of radar. An extensive list of references (Section 9) is helpful to the Reader for a deeper insight to the many interesting topics of radar.
This is the fourth part of a series of papers that provide a comprehensive survey of techniques for tracking maneuvering targets without addressing the so-called measurement-origin uncertainty. Part I and Part II deal with target motion models. Part III covers the measurement models and the associated techniques. This part surveys tracking techniques that are based on decisions regarding target maneuver. Three classes of techniques are identified and described: equivalent noise, input detection and estimation, and switching model. Maneuver detection methods are also included.
In the recent past, the capability of tracking dynamic targets from forward looking infrared (FLIR) measurements has been improved substantially by replacing standard correlation trackers with adaptive extended Kalman filters or enhanced correlator/Kalman filter combinations. This research investigates a tracker able to handle "multiple hot-spot" targets, in which digital and/or optical signal processing is employed on the FLIR data to identify the underlying target shape. Furthermore, multiple model adaptive filtering is investigated as a means of changing the field-of-view as well as the tracker bandwidth when target acceleration can vary over a wide range. The performance potential of such a tracking algorithm is shown to be substantial.
The two-stage Alpha-Beta-Gamma Estimator is proposed as an alternative to adaptive gain versions of the Alpha-Beta and Alpha-Beta-Gamma filters for tracking maneuvering targets. The purpose of this paper is to accomplish constant gain, variable dimension filtering with a two-stage Alpha-Beta-Gamma Estimator which is derived from a two-stage Kalman estimator. The noise variance reduction matrix and steady-state error covariance matrix are given as a function of the steady-state gains. A procedure for filter parameter selection is also given along with techniques for maneuver response and gain scheduled initialization.
The characterization of complex air traffic situations is an important issue in air traffic management (ATM). Within the current ground-based ATM system, complexity metrics have been introduced with the goal of evaluating the difficulty experienced by air traffic controllers in guaranteeing the appropriate aircraft separation in a sector. The rapid increase in air travel demand calls for new generation ATM systems that can safely and efficiently handle higher levels of traffic. To this purpose, part of the responsibility for separation maintenance will be delegated to the aircraft, and trajectory management functions will be further automated and distributed. The evolution toward an autonomous aircraft framework envisages new tasks where assessing complexity may be valuable and requires a whole new perspective in the definition of suitable complexity metrics. This paper presents a critical analysis of the existing approaches for modeling and predicting air traffic complexity, examining their portability to autonomous ATM systems. Possible applications and related requirements will be discussed.
The adaptive capability of the Kalman filtering is known to increase by incorporating a neural network into the normal Kalman filter. Current work extends this fact and proposes the neural network aided Kalman filtering scheme for tracking multitargets that are highly manoeuvring. The improvement in the tracking accuracy due to the proposed scheme is presented for various tracking scenarios.
This paper considers the problem of joint maneuvering target tracking and classification. Based on recently proposed Monte Carlo techniques, a multiple model (MM) particle filter and a mixture Kalman filter (MKF) are designed for two-class identification of air targets: commercial and military aircraft. The classification task is carried out by processing radar measurements only, no class (feature) measurements are used. A speed likelihood function for each class is defined using a prior information about speed constraints. Class-dependent speed likelihoods are calculated through the state estimates of each class-dependent tracker. They are combined with the kinematic measurement likelihoods in order to improve the classification process. The two designed estimators are compared and evaluated over rather complex target scenarios. The results demonstrate the usefulness of the proposed scheme for the incorporation of additional speed information. Both filters illustrate the opportunity of the particle filtering and mixture Kalman filtering to incorporate constraints in a natural way, providing reliable tracking and correct classification. Future observations contain valuable information about the current state of the dynamic systems. In the framework of the MKF, an algorithm for delayed estimation is designed for improving the current modal state estimate. It is used as an additional, more reliable information in resolving complicated classification situations.
This report describes a shape matching method which transforms a template shape represented by a tree of sigmoid functions so that it fits target shapes included in a sequence of image frames. The tree-based shape representation is automatically constructed by a training procedure in advance to the shape matching process. Through the matching process, the position and the posture of the target are computed by a gradient-descent-based searching procedure. The amount of computation is reduced by using a compact tree, which represents a shape roughly but with sufficient details, for determining the object postures. The search area is also limited assuming small changes of the object positions and postures in consecutive frames. The number of iterations in gradient-descent-based search is reduced using the previous position and posture as initial conditions for the next search. Some experiments were conducted and the method's sensitivity to noises and initial parameter values were examined. The results suggest the method's feasibility as a target tracking method.
Contenido: Sistemas de control; Diseño de sistemas modernos de control con técnicas computacionales (CAD); Motores eléctricos y sistemas de potencia; Elementos de control neural; Implementación neural FPGA; Fundamentos de lógica difusa; Fundamentos de VHDL; Estudios de caso (Corriente neural y control de velocidad en motores de inducción; Control neurodifuso de sistemas generadores sincrónicos); Apéndices.
Clustering is a fundamental data analysis method. It is widely used for pattern recognition, feature extraction, vector quantization (VQ), image segmentation, function approximation, and data mining. As an unsupervised classification technique, clustering identifies some inherent structures present in a set of objects based on a similarity measure. Clustering methods can be based on statistical model identification (McLachlan & Basford, 1988) or competitive learning. In this paper, we give a comprehensive overview of competitive learning based clustering methods. Importance is attached to a number of competitive learning based clustering neural networks such as the self-organizing map (SOM), the learning vector quantization (LVQ), the neural gas, and the ART model, and clustering algorithms such as the C-means, mountain/subtractive clustering, and fuzzy C-means (FCM) algorithms. Associated topics such as the under-utilization problem, fuzzy clustering, robust clustering, clustering based on non-Euclidean distance measures, supervised clustering, hierarchical clustering as well as cluster validity are also described. Two examples are given to demonstrate the use of the clustering methods.
A neural network recognition and tracking system is proposed for classification of radar pulses in autonomous Electronic Support Measure systems. Radar type information is considered with position-specific information from active emitters in a scene. Type-specific parameters of the input pulse stream are fed to a neural network classifier trained on samples of data collected in the field. Meanwhile, a clustering algorithm is used to separate pulses from different emitters according to position-specific parameters of the input pulse stream. Classifier responses corresponding to different emitters are separated into tracks, or trajectories, one per active emitter, allowing for more accurate identification of radar types based on multiple views of emitter data along each emitter trajectory. Such a What-and-Where fusion strategy is motivated by a similar subdivision of labor in the brain. The fuzzy ARTMAP neural network is used to classify streams of pulses according to radar type using their functional parameters. Simulation results obtained with a radar pulse data set indicate that fuzzy ARTMAP compares favorably to several other approaches when performance is measured in terms of accuracy and computational complexity. Incorporation into fuzzy ARTMAP of negative match tracking (from ARTMAP-IC) facilitated convergence during training with this data set. Other modifications improved classification of data that include missing input pattern components and missing training classes. Fuzzy ARTMAP was combined with a bank of Kalman filters to group pulses transmitted from different emitters based on their position-specific parameters, and with a module to accumulate evidence from fuzzy ARTMAP responses corresponding to the track defined for each emitter. Simulation results demonstrate that the system provides a high level of performance on complex, incomplete and overlapping radar data.
A fast target maneuver detecting and highly accurate tracking technique using a neural fuzzy network based on Kalman filter is proposed in this paper. In the automatic target tracking system, there exists an important and difficult problem: how to detect the target maneuvers and fast response to avoid miss-tracking? The traditional maneuver detection algorithms, such as variable dimension filter (VDF) and input estimation (IE) etc., are computation intensive and difficult to implement in real time. To solve this problem, neural network algorithms have been issued recently. However, the normal neural networks such as backpropagation networks usually produce the extra problems of low convergence speed and/or large network size. Furthermore, the way to decide the network structure is heuristic. To overcome these defects and to make use of neural learning ability, a developed standard Kalman filter with a self-constructing neural fuzzy inference network (KF-SONFIN) algorithm for target tracking is presented in this paper. By generating possible target trajectories including maneuver information to train the SONFIN, the trained SONFIN can detect when the maneuver occurred, the magnitude of maneuver values and when the maneuver disappeared. Without having to change the structure of Kalman filter nor modeling the maneuvering target, this new algorithm, SONFIN, can always find itself an economic network size with a fast learning process. Simulation results show that the KF-SONFIN is superior to the traditional IE and VDF methods in estimation accuracy.
Both simultaneous localization and mapping (SLAM) and detection
and tracking of moving objects (DTMO) play key roles in robotics and
automation. For certain constrained environments, SLAM and DTMO are
becoming solved problems, but for robots working outdoors and at high
speeds, SLAM and DTMO are still incomplete. In earlier works, SLAM and
DTMO are treated as two separate problems. In fact, they can be
complementary to one another. In this paper, we present a new method to
integrate SLAM and DTMO to solve both problems simultaneously for both
indoor and outdoor applications. The results of experiments carried out
with CMU Navlab8 and Navlab11 vehicles with the maximum speed of 45 mph
in crowded urban and suburban areas verify the described work