PreprintPDF Available

A Hybrid Model and Learning-Based Adaptive Navigation Filter

Authors:
  • Google and Reichman Tech School
Preprints and early-stage research may not have been peer reviewed yet.

Abstract and Figures

The fusion between an inertial navigation system and global navigation satellite systems is regularly used in many platforms such as drones, land vehicles, and marine vessels. The fusion is commonly carried out in a model-based extended Kalman filter framework. One of the critical parameters of the filter is the process noise covariance. It is responsible for the real-time solution accuracy, as it considers both vehicle dynamics uncertainty and the inertial sensors quality. In most situations, the process noise is covariance assumed to be constant. Yet, due to vehicle dynamics and sensor measurement variations throughout the trajectory, the process noise covariance is subject to change. To cope with such situations, several adaptive model-based Kalman filters were suggested in the literature. In this paper, we propose a hybrid model and learning-based adaptive navigation filter. We rely on the model-based Kalman filter and design a deep neural network model to tune the momentary system noise covariance matrix, based only on the inertial sensor readings. Once the process noise covariance is learned, it is plugged into the well-established, model-based Kalman filter. After deriving the proposed hybrid framework, field experiment results using a quadrotor are presented and a comparison to model-based adaptive approaches is given. We show that the proposed method obtained an improvement of 25% in the position error. Furthermore, the proposed hybrid learning method can be used in any navigation filter and also in any relevant estimation problem.
Content may be subject to copyright.
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 1
A Hybrid Model and Learning-Based
Adaptive Navigation Filter
Barak Or, Member, IEEE, and Itzik Klein, Senior Member, IEEE
Abstract—The fusion between an inertial navigation system
and global navigation satellite systems is regularly used in many
platforms such as drones, land vehicles, and marine vessels.
The fusion is commonly carried out in a model-based extended
Kalman filter framework. One of the critical parameters of the
filter is the process noise covariance. It is responsible for the
real-time solution accuracy, as it considers both vehicle dynamics
uncertainty and the inertial sensors quality. In most situations
the process noise is covariance assumed to be constant. Yet,
due to vehicle dynamics and sensor measurement variations
throughout the trajectory, the process noise covariance is subject
to change. To cope with such situations, several adaptive model-
based Kalman filters were suggested in the literature. In this
paper, we propose a hybrid model and learning-based adaptive
navigation filter. We rely on the model-based Kalman filter and
design a deep neural network model to tune the momentary
system noise covariance matrix, based only on the inertial sensor
readings. Once the process noise covariance is learned, it is
plugged into the well-established, model-based Kalman filter.
After deriving the proposed hybrid framework, field experiment
results using a quadrotor are presented and a comparison to
model-based adaptive approaches is given. We show that the
proposed method obtained an improvement of 25% in the po-
sition error. Furthermore, the proposed hybrid learning method
can be used in any navigation filter and also in any relevant
estimation problem.
Index Terms—Adaptive Algorithm, Deep Learning, Global
Navigation Satellite System, Inertial Measurement Unit, Inertial
Navigation Systems, Kalman Filter, Machine Learning, Quad-
copter, Supervised Learning, Unmanned Autonomous Vehicles,
Vehicle Tracking.
I. INTRODUCTION
AUTONOMOUS vehicles, such as autonomous under-
water vehicles (AUV) or quadrotors, are commonly
equipped with an inertial navigation system (INS) and other
sensors [1] to provide real-time information about their posi-
tion, velocity, and orientation [2]–[7]. The INS has two types
of inertial sensors; namely, gyroscopes and accelerometers.
The former measures the angular velocity vector, and the latter
measures the specific force vector. The inertial sensors are
combined in an inertial measurement unit (IMU). Their use for
navigation is challenging due to their high noise levels, which
eventually accumulate, leading to a navigation solution drift
over time. To solve this problem, an external aiding sensor,
such as the global navigation satellite system (GNSS) receiver,
is regularly used in a a navigation filter [2], [3], [6]. Such
fusion is carried out using a nonlinear filter, where the error
state implementation of the extended Kalman filter (es-EKF)
is usually employed [1], [8]. In the filter, the process noise
Barak Or and Itzik Klein are with the Department of Marine Technologies,
Charney School of Marine Science, University of Haifa, Haifa, 3498838, Israel
(e-mail: barakorr@gmail.com, kitzik@univ@haifa.ac.il).
covariance matrix is based on the IMU measurements char-
acteristics and/or vehicle dynamics while the external aiding
measurements specification determines the measurement noise
covariance matrix. Those two covariances matrices have a
major influence on the filter performance. A common practice
is to assume that both covariances are fixed during the vehicle
operation. However, as the sensor characteristics are subject to
change during operation and physical constraints do not allow
capturing the vehicle dynamic optimally, those covariances
should vary over time as the amount of uncertainty varies and
is unknown. Thus, tuning the process or measurement noise
covariances optimally can lead to a significant improvement
in the filter performance. [8].
To cope with this problem, several model-based attempts
have been made in the literature to develop optimal adaptive
filters [9]–[15]. The most common among them is based on
the filter innovation process [16], where a measure of the
new information is calculated at every iteration, leading to
an update to the process noise covariance matrix. Still, the
question of the optimal approach to tune the system noise
covariance matrix is considered open and is addressed in detail
in [17].
Recently, deep learning approaches were integrated in
model based pedestrian dead-reckoning (PDR) algorithms.
One of the initial works in the field is the robust IMU double
integration, RIDI approach [18], where neural networks were
trained to regress linear velocities from inertial sensors to
constrain the accelerometer readings, although it was the first
work in the field, double integration is still needed and the
error accumulates rapidly. In [19], the device orientation, in
addition to the accelerometers and gyroscopes readings, was
also used as input to a deep learning architecture to regress the
user velocity in 2D. The velocity is then integrated to obtain
the user position. More pedestrian inertial navigation examples
are described by [20] where recent databases, methods and
real-time inferences can be found. Later, PDRNet [21], utilizes
a smartphone location recognition classification network fol-
lowed by a change of heading and distance regression network
to estimate the used 2D position. One of the main challenges
in model or learning based PDR is the estimation of the user
walking direction as the device is not always aligned with the
user motion. To cope with that, in [22] a novel DNN structure
was designed for extracting the motion vector in the device
coordinates, using accelerometer readings.
In AUV navigation, an end-to-end deep learning approach
was proposed to regress missing Doppler velocity log (DVL)
beam measurements to provide the AUV velocity vector, only
when a single beam is missing. [23]. Later, [24], a deep
learning approach was used to address with a DVL failure
arXiv:2207.12082v1 [eess.SY] 14 Jun 2022
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 2
scenarios to predict the DVL output.
For land vehicle navigation a deep neural network (DNN)
model was presented to overcome the inaccurate dynamics
and observation models of wheel odometry for localization,
named RINS-W [25]. Their novel approach exploited DNNs
to identify patterns in wheeled vehicle motions using the IMU
sensor only. To overcome the navigation solution drift in pure
inertial quadrotor navigation, QuadNet, a hybrid framework to
estimate the quadrotor’s three-dimensional position vector at
any user-defined time rate was proposed [26].
Focusing on the estimation process, a self-learning square
root-cubature Kalman filter was proposed [27]. This hybrid
navigation filter enhanced the navigation accuracy by provid-
ing observation continuously, even in GNSS denied environ-
ments, by learning transfer rules of internal signal in the filter.
In [28], recurrent neural networks are employed to learn the
vehicle’s geometrical and kinematic features to regress the
process noise value in a linear Kalman filer framework. In
[29], DNN based multi models (triggered by traffic conditions
classification) together with an extended Kalman filter was
proposed to cope with GNSS outages. Recently, a DNN
model was proposed to solve the GNSS outages, mainly for
unmanned air vehicles (UAV). In [30], a convolutional neural
network model was used for noise-free gyro measurements in
open loop attitude estimation. They obtained state-of-the-art
navigation performance in terms of attitude estimation where
they compensated for gyro measurement errors as part of
a strapdown integration approach. This trend of integrating
learning approaches in various fields and applications also
raises the motivation to adopt such approaches in the described
problem.
In this paper, we propose a hybrid model and learning-
based adaptive navigation filter. We rely on the model-based
extended Kalman filter; yet, instead of using model-based
adaptive process noise tuning, we design a DNN to tune the
momentary system noise covariance matrix, based only on the
inertial sensor readings. Once, the process noise covariance
is learned, it is plugged into the well-established, model-
based Kalman filter. To that end, we simulated many vehicle
trajectories based on six baseline trajectories to create a rich
dataset. This dataset contains rich dynamic scenarios with
many motion patterns and different IMU noise covariances.
This rich dataset was created to enable robustness to motion
dynamics and IMU characteristics. Based on this dataset, we
adopt a supervised learning (SL) approach to regress the
process noise covariance.
The main contributions of this paper are:
1) Derivation of an adaptive hybrid learning algorithm to
momentary determine the process noise covariance matrix
as a function of the IMU measurements. That is, instead
of using model-based approaches (as been done in the
last 50 years), to the best of our knowledge, we are the
first to propose an adaptive hybrid learning algorithm.
2) Online integration of the proposed hybrid learning al-
gorithm with es-EKF implementation of the navigation
filter. In that manner, the proposed approach can be used
with any external sensor aiding the INS.
3) The proposed approach principles can be further applied
to any estimation problem that requires adaptive tuning
of the process noise covariance.
The advantages of the proposed approach lies in its hybrid
fusion of the well celebrated EKF and applying learning
approaches on the EKF soft spot, the adaptive determination
of the process noise covariance. By using learning approaches
we leverage from their ability to generalization of intrinsic
properties appearing in sequential datasets and, therefore,
better cope with varying conditions affecting the process noise
value.
After deriving and validating the proposed framework using
the stimulative dataset, field experiments using a quadrotor
are done to evaluate the robustness and performance of the
proposed approach testing phase and compare it to other adap-
tive model-based approaches. The recordings from the field
experiment are used as a test dataset and also to examine our
network capability for generalization. In our setup, we used
GNSS position measurements to update the INS; however, the
proposed approach is suitable for any aiding sensor and for
any platform.
The rest of the paper is organized as follows: Section II
presents the problem formulation for the INS/GNSS fusion in
an es-EKF implementation with constant and adaptive process
noise covariance. Section III presents our proposed hybrid
adaptive EKF. Section IV gives the results, and Section V
presents the conclusions of this study.
II. ADA PT IV E NAVIG ATIO N FILTER
The nonlinear nature of the INS equations requires a nonlin-
ear filter. The most common filter for fusing INS with external
aiding sensors is the EKF [1], and in particular, with a 15 error
states implementation:
δx=δpnδvnδεnbabgTR15×1,(1)
where δpnR3×1is the position vector error states expressed
in the navigation frame, δvnR3×1is the velocity vector
error states expressed in the navigation frame, δεnR3×1is
the misalignment vector expressed in the navigation frame,
baR3×1is the accelerometer bias residuals vector ex-
pressed in the body frame, and bgR3×1is the gyro bias
residuals vector expressed in the body frame. The navigation
frame is denoted by nand the body frame is denoted by b. The
navigation frame center is located at the body center of mass,
where the x-axis points to the geodetic north, z-axis points
down parallel to local vertical, and y-axis completes a right-
handed orthogonal frame. The body frame center is located
at the center of mass, x-axis is parallel to the longitudinal
axis of symmetry of the vehicle pointing forwards, the y-axis
points right, and the z-axis points down such that it forms
a right-handed orthogonal frame. The linearized, error-state,
continuous-time model is
δ˙x =Fδx+Gδw,(2)
where FR15×15 is the system matrix, GR15×12 is
the shaping matrix, and δw=wawgwab wgb T
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 3
R12×1is the system noise vector consisting of the accelerom-
eter, gyro, and their biases random walk noises, respectively
[8]. The system matrix, F, is given by
F=
Fpp Fpv F 03×303×3
Fvp Fvv Fvε Tn
b03×3
Fεp Fεv Fεε 03×3Tn
b
03×303×303×303×303×3
03×303×303×303×303×3
,(3)
where Tn
bis the transformation matrix between body and
navigation frame and Fij R3×3can be found explicitly
in the literature (see [1], [31], [32]). The shaping matrix is
given by
G=
Tn
b03×303×303×3
03×3Tn
b03×303×3
03×303×303×303×3
03×303×3I3×303×3
03×303×303×3I3×3
,(4)
where I3×3is an identity matrix.
The corresponding discrete version of the navigation model
(for small step sizes), as given in (2), is
δxk+1 =Φkδxk+Gkδwk.(5)
The transition matrix, Φk, is defined by a first order approxi-
mation as
Φk
=I+Ft, (6)
where kis a time index, tis the time step size, Fis the
system matrix (obtained using (3) with the estimated state
vector at time k), and δwkis a zero mean white Gaussian
noise. The discretized process noise is given by
Qd
k=GQcGTt, (7)
where Gis defined in (4) (using the current estimated state
vector at time k), and Qcis the continuous process noise
matrix.
The discrete es-EKF, as described here, is used in the fusion
process between the INS and the external measurements.
Equations (8)(12) summarize the navigation filter equations
[1], [31]:
δˆx
k=0(8)
P
k=Φk1Pk1Φk1T+Qd
k1(9)
Kk=P
kHkThHkP
kHkT+Rd
ki1
(10)
δˆxk=Kkδzk(11)
Pk= [IKkHk]P
k,(12)
where δˆx
kis the prior estimate of the error state and δˆxk
is the posterior estimate, with their corresponding covariance
matrices P
k(9) and Pk(12), respectively, Kk(10) is the
Kalman gain, δzkis the kmeasurement residual, Hkis
the measurement matrix, and Rkis the measurement noise
covariance, assumed known and constant.
A. Measurement Model
The GNSS position measurements are available in a con-
stant and lower frequency then that of the IMU. After pro-
cessing, the GNSS receiver outputs the vehicle position vector
in the navigation frame, where it is used as an update in
the navigation filter. Hence, the corresponding time-invariant
GNSS measurement matrix is given by
HGNS S =I3×303×12 R3×15.(13)
The corresponding measurement residual is given by
δzGNS S ,j =HGN SS δxj+ςGN S S ,j ,(14)
where ςGNS S ,j N 0,RdGN SS R3×1is an additive,
discretized, zero mean white Gaussian noise. It is assumed
that ςjand δwjare uncorrelated.
B. Model Based-Adaptive Noise Covariance
The es-EKF minimizes the tracking error without changing
Qmatrix online; that is, the process noise covariance is
constant throughout the operation. As shown in the literature,
[17] and the references therein, tuning Qonline greatly
improves the navigation filter performance .
For simplicity, we assume Qd
kis a diagonal matrix, thus no
correlation exists between the noise terms.
The optimal diagonal Qmatrix at time step kis defined by
Q
k
= arg min
Qk∈Q
ˆxk(Qk)xGT
k
2
2,(15)
where Qis the admissible set of Qkat time index k,xGT
kis
the ground truth (GT) state at time index k, and k·k2is the
second Euclidean norm.
The most common approach to estimate Qin an adaptive EKF
framework was suggested in [16], [17], and is based on the
innovation matrix for a window of size ξ:
Ck
=1
ξ
k
X
i=kξ+1
νiνiT,(16)
where Ckis the innovation matrix and the innovation vector
is defined by
νk
=zkx
k.(17)
The innovation matrix, (16), is used together with the Kalman
gain, K, to adapt Q, as follows:
ˆ
Qk=KkCkKT
k.(18)
III. A HYBRID ADAP TI VE NAVI GATI ON FI LTER
We rely on the well established, model-based EKF, yet
instead of a model-based adaptive approach, a data-driven
approach is used to regress the momentary process noise
covariance, and, as a result, creates a hybrid adaptive EKF.
To this end, the navigation filter equations (8)-(12) are used,
but instead of using a constant process noise (7) or a model-
based adaptive process noise (18), we derive a data-driven ap-
proach to estimate the time-varying continuous process noise
covariance matrix Qc. Our proposed hybrid adaptive EKF
framework, suitable for any aided INS scenario, is presented
in Figure 1.
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 4
Fig. 1. A hybrid adaptive navigation filter framework with online tuning of
the process noise covariance matrix. .
A. Data-Driven Based Adaptive Noise Covariance
It is assumed that the continuous process noise covariance
matrix, Qc
k, at time k, is a diagonal matrix with the following
structure:
Qc
k=
diagnqx
f, qy
f, qz
f, qx
ω, qy
ω, qz
ω, εI1×6okR12×12,(19)
where the variance for each of the accelerometer axes is given
by qi
fand for the gyroscope by qi
ω(ix, y, z). The biases
are modeled by random walk processes, and their variances are
set to ε= 0.001. By taking this approach, our goal is to ease a
machine learning ML algorithm for accurately regressing the
six unknown variances in (19) and thereby claim that these
are the dominant variances needed in an adaptive framework.
We define a general one-dimensional series of length Nof
the inertial sensor (accelerometer or gyroscope) readings in a
single axis by
Sk={si}k1
i=kN.(20)
The discussed optimization problem is to find q
ksuch that
the position error is minimized. By doing that for all six IMU
channels, the es-EKF considers an optimal system noise co-
variance matrix and provides improved tracking performance.
The power of ML increases the ability to solve many difficult
and non-conventional tasks. To determine Q
kin (15), the
problem is formulated as a SL problem. Formally, we search
for a model to relate an instance space, X, and a label space,
Y. We assume that there exists a target function, F, such that
Y=F(X).
Thus, the SL task for adaptive determination of the process
noise covariance is to find F, given a finite set of labeled
examples, inertial sensor readings, and corresponding process
noise variance values:
{Sk, q
k}M
k=1 .(21)
The SL approach aims to find a function ˜
Fthat best estimates
F. To that end, for the training process, a loss function, l, is
defined to quantify the quality of ˜
Fwith respect to F. The
loss function is given by
LY,ˆ
Y(X)
=1
M
M
X
m=1
l(y, ˆy)m,(22)
where Mis the number of examples, and mis the example
index. Minimizing Lin a training/test procedure leads to the
target function. In our problem, the loss function is given by
lm
= (q
mˆqm)2,(23)
where ˆqmis the estimated term obtained by the learning model
during the training process.
In this work, the train dataset used to find the function ˜
Fgiven
the labeled examples (21), as described in the next section.
B. Dataset Generation
There are several existing inertial datasets as were recently
summarized in [33]. Yet, none of those datasets fits our
problem as they do not provide enough IMU readings with
different noise characteristic. As a consequence, we generated
are own dataset using a simulation (detailed in Appendix B).
To generate the dataset six different baseline trajectories
were simulated as presented in Figure 2(a). The richness
of the trajectories, as a result of their diversity, allows the
establishment of a model to cope with unseen trajectories.
Each baseline trajectory was created by generating ideal IMU
readings for a period of 400s in a sampling rate of 100Hz,
resulting in a sequence of 6×40,000 samples for each baseline
trajectory as shown in Figure 2.
Our task is to estimate the system noise covariance matrix, or
specifically the noise variance of each IMU channel. Hence,
we divided up each of the six IMU channels and corrupted
their perfect data with additive white Gaussian noise with
variance in the range of q[0.001,0.025], with 15 different
values inside this interval, as shown in Figure 2(b).
Hence, each baseline trajectory has 15 series of 6×40,000
noisy inertial samples, as shown in Figure 2(b). The justifi-
cation to include a simple noise model lies in the momentary
IMU measurement noise covariance sequence. A short time
window for the IMU measurements is considered and allows
characterizing the noise with its variance only. Next, the series
length Nis chosen, and a corresponding labeled database is
generated as shown in Figure 2(c). Lastly, we choose N= 200
samples and create batches corresponding to two seconds each
with a total of 6×200×15. Then, these batches are randomly
divided into train and test datasets in such a manner that all
baseline trajectories are included in the train and the test sets
in a ratio of 80:20, as described in Figure 2(d).
C. Architecture design
Several architectures were designed and evaluated. They
include recurrent DNN, and, in particular, the long short-term
memory (LSTM) architectures for time series data. However,
in our problem they did not perform well and failed in
generalization of the noise variance property. One of the
architecture we examined included a bi-directional LSTM
layer followed by two linear layers. The training failed as
the temporal information wasn’t relevant for this task. On
the other hand, an architecture with only convolution layers
achieved satisfactory results, where it minimized the test loss
very fast and obtained a low position root mean squared
error (PRMSE) (the metric is defined in (25)).This result are
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 5
Fig. 2. Dataset generation and pre-processing phase: (a) six different simulated baseline trajectories. Each baseline trajectory was created by tuning the IMU
signals a period of 400[s]at a frequency of 100[Hz], resulting in a series of 6×40,000 samples for each baseline trajectory, obtained in phase (b). Example
generation (c): given series length, N, examples are created and stored with their label. (d) split the database into train set and test set with a ratio of 80:20.
Fig. 3. Our six baseline trajectories.
due to the spatial information the noisy IMU signals have.
This information structure was easily captured by the spatial
convolutional operates in the convolution layers. As the goal
of this paper is to lay the foundations for a hybrid, adaptive
navigation filter approach, we describe here only the DNN
architecture as presented in Figure 4. Our DNN model has the
following layers:
1) Linear layer: This is the simplified layer structure that
applies a linear transformation to the incoming data from
the previous layer. The input features are received in the
form of a flattened 1D vector and are multiplied by the
weighting matrix.
2) Conv1D layer: A convolutional, one-dimensional
(Conv1D) layer creates a convolution kernel that is
convolved with the layer input over a single spatial
dimension to produce a vector of outputs. During the
training phase the DNN learns the optimal weights
of the kernels. In our DNN model the input layer is
followed by a chain of three Conv1D layers. There are
five kernels for the first layer and three kernels for the
second and third layers. The kernel size decreases as
the inputs go deep: the first Conv1D layer has twenty
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 6
kernels, the second Conv1D layer has ten kernels, and
the third Conv1D layer has five kernels.
3) Global average pooling layer: Pooling layers help with
better generalization capability as they perform down
sampling feature map. In this way, the robustness of the
DNN grows. For example, better handling with changes
in the position of the features inside the 1D input vector.
For the DNN architecture we selected the global average
pooling method, which calculates the average for each
input channel and flattens the data so it passes from
Conv1D through the linear layers.
4) Leaky ReLU: Leaky rectified linear unit [34] is a non-
linear activation function obtaining a value αwith the
following output:
f(α) = α α > 0
0.01α α 0.(24)
The main advantage of the leaky ReLU over the classical
ReLU (0 for α0) is the small positive gradient when
the unit is not active, which deals better with the dying
ReLU problem [35]. The leaky ReLU functions were
combined after each layer in the DNN model.
5) Layer normalization: One of the challenges of DNN
is that the gradients with respect to the weights in one
layer are highly dependent on the outputs of the neurons
in the previous layer, especially if these outputs change
in a highly correlated way [36]. Batch normalization, and
in particular layer normalization, was proposed to reduce
such an undesirable covariate shift. The layer normaliza-
tion was added after every Conv1D layer (together with
the leaky ReLU layer).
The parameters of the above layers are as follow: a series of
200×1samples is inserted to a one-dimensional convolutional
layer (Conv1D) with a kernel size equal to 20 with ve filters.
Then, a leaky ReLU activation function and normalization
layer are applied, adding nonlinearity for better generalization
capability of the DNN. We duplicated this structure twice
more with three filters each and smaller kernels: the second
with a kernel size equals to ten and the third with a kernel
size equals to five. The output is inserted to a global average
pooling layer followed by four linear layers, with leaky ReLU
activation functions between them, with 100,80,50 and 20
weights, retrospectively. Finally, the DNN outputs the process
noise covariance, q.
D. Hybrid Learning Approach
In this work, GNSS position updates are employed to
aid the INS and demonstrate our hybrid learning approach.
Applying the suggested adaptive tuning approach in online
setting involves integrating the INS/GNSS with the regressor
as presented in Figure 5. Algorithm 1 describes the INS/GNSS
with process noise covariance learning in a real time setup.
The IMU signals are inserted both into the INS/GNSS filter
(8) (12) and the DNN model. The regressed continuous
process noise covariance is plugged into (7) to obtain the
discrete one, which in turn is substituted into (9).
Algorithm 1 Hybrid adaptive filter applied to INS/GNSS
Input: ωib,fb,vAiding ,t0,τ, T, tuning Rate
Output: vn, εn
Initialization :vn
0, εn
0
LOOP Process
1: for t= 0 to Tdo
2: obtain ωib,fb
3: solve navigation equations (3)
4: if (mod(t, τ)=0) then
5: obtain pGNS S
6: update navigation state using the es-EKF (8)-(14)
7: end if
8: Calculate DNN model and predict Qc
k+1.
9: if mod(t, tuningRate) = 0 then
10: Qc
k+1 =˜
FNN (Sk)
11: end if
12: end for
IV. ANA LYSI S AN D RES ULTS
We employ the following two metrics to evaluate the
position accuracy of the proposed approach:
Position root mean squared error (2nd norm) for all three
axes:
P RM SE =v
u
u
t
1
c
c
X
k=1 X
j∈{x,y,z}
δˆp2
jk .(25)
Position mean absolute error:
P M AE =1
c
c
X
k=1 X
j∈{x,y,z}
|δˆpjk |.(26)
where c is the number of samples, kis a running index for the
samples, jis the running index for the Cartesian coordinate
system, and δˆpjk is the position error term.
A. Train Dataset Analysis
The proposed DNN architecture, presented in Section III-C,
was trained on the dataset described in Section III-B. After 30
epochs it achieved an RMSE of 0.0032 on the test set, show-
ing the proposed DNN generalization capability and thereby
establishing the trained DNN regressor model. The training
process was done in mini-batches of 500 examples each. The
ADAM optimizer [37] was used with an initial learning rate
of 0.001 and a learn-rate-schedule with a drop factor of 0.1
after 20 epochs. In our working environment (Intel i7-6700HQ
CPU@2.6GHz 16GB RAM with MATLAB), the training time
for input length of 200 samples elapsed about three hours.
The averaged inference time is 0.02[s]). Figure 6 presents
the performance for N= 200 samples where the red line
is the desired one, representing when the GT value is equal
to the DNN predicted value. The gray points represent the
performance of the test set with the trained DNN model. The
mean values of the test set lies near the red line (blue points)
for most cases.
The INS/GNSS fusion is mostly performed under real-
time conditions, where latency in the position computation
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 7
Fig. 4. DNN architecture: Three Conv1D layers are followed by four linear layers and a global average pooling layer. Leaky ReLU activation functions are
used after every layer to add nonlinearity and achieve better generalization capability.
Fig. 5. Our adaptive hybrid adaptive filter applied to the INS/GNSS fusion
process to tune the process noise covariance matrix, ˆ
Qk. Both IMU and
GNSS measurements are plugged into the INS/GNSS filter, while the six
IMU channels are also inserted into the DNN for predicting the process noise
covariance matrix.
Fig. 6. GT values versus predicted DNN values on the test set. All values
are presented by gray crosses. The blue mean points represent the mean for
each value, and the dashed red line (true value=predict value) represents the
DNN generalization capability.
might degrade the performance. Thus, the influence of the
IMU sequence length input was examined. The system noise
covariance matrix was learned, based on series of length N. As
N >> 1, the probability of learning the correct terms grows,
since the DNN can capture the signal intrinsic properties easier
using more data; however, the latency grows. Considering this
trade-off, we trained the chosen architecture with various N
values and calculated the RMSE as shown in Figure 7. As
expected, N= 400 obtained the minimum RMSE, yet for the
rest of the analysis we chose N= 200 (similar RMSE) to
receive the regression result as it is given in a shorter time
period. For example, in the stimulative train and test dataset
the sampling rate is 100Hz, thus working with N= 200 gives
the regression result every two seconds instead of working
with four seconds (N= 400).
Fig. 7. DNN RMSE vs. input length (N).
B. Field experiment
A field experiment with the DJI Matrice 300 [38] was
performed (Figure 8). The trajectory is shown in Figure 9.
An alike figure-eight flight trajectory was made with some
additional approximated straight segments. The experiment
parameters, including the GNSS RTK GT values and IMU,
are given in Table I. The quadrotor dataset was published
in [39] and is publicly available at https://github.com/ansfl/
Navigation-Data-Project.
Equations (2)-(12) were used as a model-based EKF with
a constant process noise covariance matrix. Three different
cases of the model-based EKF with constant process noise
covariance matrix (MB-EKF-CQ) were examined:
MB-EKF-CQ1: q1:3 = 0.002, q4:6 = 0.02.
MB-EKF-CQ2: q1:3 = 0.001, q4:6 = 0.01.
MB-EKF-CQ3: q1:3 = 0.002, q4:6 = 0.01.
In addition, using (16)-(18), three different (window size
length) cases of model-based EKF with the adaptive process
noise covariance matrix (MB-EKF-AQ) were examined:
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 8
Fig. 8. DJI matrice 300 in our field experiment.
Fig. 9. The trajectory of the quadrotor used to evaluate the proposed approach.
MB-EKF-AQ1: ξ= 1.
MB-EKF-AQ2: ξ= 3.
MB-EKF-AQ3: ξ= 5.
All six approaches were compared to our proposed hybrid
adaptive EKF (HB-AEKF) using the trained network, as
described in the previous section. Results in terms of PMAE
and PRMSE are presented in Table 2. The MB-EKF-AQ
model-based adaptive filters obtained better performance than
the MB-EKF-CQ constant process noise filters. In particular,
MB-EKF-CQ3 achieved the best performance for constant
process noise with a PMAE of 2.2[m]and a PRMSE of
1.6[m], where MB-EKF-AQ3 manged to improve them by
18% and 19%, respectively. Our proposed hybrid adaptive
approach, HB-AEKF, with an input length of 200 samples,
obtained the best performance, with 1.7[m]PMAE and
1.2[m]PRMSE improving MB-EKF-AQ3 by 5.5% and 7.7%,
respectively.
The qparameters time evaluation were obtained using
our HB-AEKF approach—are shown in Figure 10 for both
accelerometers and gyroscopes. Their behavior shows fluctua-
tions in the first 25 seconds and then steady-state behavior.
This behavior can be attributed to the convergence of the
Kalman filter (8)-(12) once it obtains enough data and cap-
tures the IMU noise statistic.
TABLE I
EXP ERI ME NT PAR AME TER S
Description Symbol Value
GNSS noise (var) - horizon R11, R22 0.01[m]2
GNSS noise (var) - vertical R33 0.02[m]2
GNSS step size τ0.2[s]
IMU step size t0.001[s]
Accelerometer noise (MPU-9250) q
1:3 0.022[m/s]2
Gyroscope rate noise (MPU-9250) q
4:6 0.0022[rad/s]2
Experiment duration T60[s]
Initial velocity vn
0[0,0,0]T[m/s]
Initial position pn
0[32.10,34.80,0]T
TABLE II
EXP ERI ME NT RE SULT S SHO WIN G PMAE AN D PRMSE PERFORMANCE
Approach PMAE[m]PRMSE[m]
HB-AEKF (ours) 1.7 1.2
MB-EKF-AQ1 2.3 1.6
MB-EKF-AQ2 2.0 1.4
MB-EKF-AQ3 1.8 1.3
MB-EKF-CQ1 2.4 1.8
MB-EKF-CQ2 2.4 1.7
MB-EKF-CQ3 2.2 1.6
Figure 11 summarizes the positioning performance by provid-
Fig. 10. The process noise parameters as a function of time as obtained
using our HB-AEKF approach. The upper plot shows the accelerometer noise
variance for each axis as detected by the DNN model, and the lower plot
shows the same for the gyroscope.
ing two graphs of the cumulative density function (CDF); one
representing PRMSE and the other the PMAE. In those plots,
all three MB-EKF-AQ and three MB-EKF-CQ approaches are
presented as well as our proposed HB-AEKF hybrid approach
(black lines). Both graphs show clearly that the entire error is
lower for HB-AEKF than all other approaches. The PRMSE of
the MB-AEKF is lower than 1.7[m]and the PMAE lower than
2.8[m]for the entire scenario. In comparison with the MB-
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 9
EKF-AQ1 approach, the maximum PRMSE and the maximum
PMAE might grow too much: 3.7[m]and 4.0[m], respectively.
Fig. 11. PRMSE and PMAE CDF graphs.
V. CONCLUSIONS
The proper choice for the system noise covariance matrix
is critical for accurate INS/GNSS fusion. In this paper a
hybrid learning and model based approach was suggested
in an adaptive filter framework. To that end, a novel DNN-
based approach to learn and online tune the system noise
covariance was developed. The input to the DNN is only the
IMU readings. This learning approach was then combined with
the model based es-EKF, resulting in a hybrid adaptive filter.
The DNN model has been trained only once on a simulated
database consisting of six different baseline trajectories. Anal-
ysis on this dataset showed a trade-off between accuracy and
the regression solution latency, where as the latency increases
the accuracy decreases. Relying on this analysis, an input
length of 200 samples was chosen for the analysis.
To validate the performance of the proposed approach, it
was compared to six model-based approaches representing
both constant and adaptive process noise covariance selection.
Quadrotor field experiments were conducted and used as an
additional test dataset on the trained network. We obtained
state-of-the-art results for each noise value compared to clas-
sical adaptive KF approaches; a PRMSE lower than 1.2[m]
and a PMAE lower than 1.7[m]were obtained using our
suggested approach, compared to a PRMSE of 1.6[m]and
a PMAE of a 2.3[m]that were obtained using the innovation-
based adaptive approach and classical tuning. Results in a
total PRMSE improvement of 24% and PMAE improvement
of 27%.
This improvement also demonstrates the robustness and gen-
eralization properties of the proposed DNN architecture, en-
abling it to cope with unseen data from different IMU sensors.
In addition, and together with the train dataset results, it proves
our hypothesis of regressing only the first six elements in the
diagonal of the continuous process noise covariance.
Although demonstrated for quadrotor INS/GNSS fusion, the
proposed approach can be elaborated for any external sensor
aiding the INS and for any type of platform.
APPENDIX
A. Abbreviations
TABLE III
ABBREVIATION AND DESCRIPTION
Abbreviation Description
AUV autonomous underwater vehicle
CDF cumulative density function
Conv1D one-dimensional convolution
DNN deep neural network
DVL Doppler velocity log
EKF extended Kalman filter
es error state
GNSS global navigation satellite system
GT ground truth
HB-AEKF Hybrid Based Adaptive EKF
INS inertial navigation system
LSTM long short-term memory (LSTM)
MB-EKF-AQ Model Based EKF Adaptive Q
MB-EKF-CQ Model Based EKF Constant Q
ML machine learning
NED North-East-Down
PDR pedestrian dead-reckoning
PMAE position mean absolute error
PRMSE position RMSE
RMSE root mean squared error
SL supervised learning
UAV unmanned air vehicles
B. Dataset Generation
1) INS Equations of Motion: The INS equations of motion
include the rate of change of the position, velocity, and the
transformation between the navigation and body frame, as
shown in Fig.12.
The position vector is given by
pn=φλhTR3×1,(27)
where φis the latitude, λis the longitude, and his the altitude.
The velocity vector is Earth referenced and expressed in the
North-East-Down (NED) coordinate system:
vn=vNvEvDTR3×1,(28)
where vN, vE, vDdenote the velocity vector components in
north, east, and down directions, respectively. The rate of
change of the position is given by [1]
˙pn=
˙
φ
˙
λ
˙
h
=
vN
RM+h
vE
cos(φ)(RN+h)
vD
,(29)
where RMand RNare the meridian radius and the normal
radius of curvature, respectively. The rate of change of the
velocity vector is given by [1]
˙vn=Tn
bfb+gn([ωn
en×] + 2 [ωn
ie×]) vn,(30)
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 10
where Tn
bR3×3is the transformation matrix from body
frame to the navigation frame. fbR3×1is the accelerometers
vector expressed in the body frame, gnR3×1is the gravity
vector expressed in the navigation frame. ωn
en is the angular
velocity vector between the earth centered earth fixed (ECEF)
frame and the navigation frame. The angular velocity vector
between ECEF and the inertial frame is given by ωn
ie and the
rate of change of the transformation matrix is given by [1]
˙
Tn
b=Tn
bωb
ib×ωb
in×,(31)
where ωb
ib =p q r TR3×1is the angular velocity
vector as obtained by the gyroscope and ωb
in is the angular
velocity vector between the navigation frame and the inertial
frame expressed in the body frame. The angular velocity
between the navigation frame and the inertial frame expressed
in the navigation frame is given by ωn
in. The alignment
between body frame and navigation frame can be obtained
from Tn
b, as follows
ε=
ϕ
θ
ψ
=
atan2Tb
n31,Tb
n32
arccos Tb
n33
atan2Tb
n13,Tb
n23
R3×1,(32)
where ϕis the roll angle, θis the pitch angle, and ψis the
yaw angle. These three angles are called Euler angles.
Fig. 12. Autonomous underwater vehicle and quadcopter with their body axes
and angular rates.
2) IMU readings: The GT trajectories were created by
tuning vehicle angular rates, P, and vehicle accelerations, a,
as follows [40]
P=˙ϕ˙
θ˙
ψTR3×1,(33)
and,
a=axayazTR3×1.(34)
The transformation between the desired motion and the INS
framework for the gyroscope readings is given by [40]
ωb
ib =Tb
n(ωn
en +ωn
ie) + ωb
ib (35)
where
ωb
ib =
1 0 sin θ
0 cos ϕsin ϕcos θ
0sin ϕcos ϕcos θ
P R3×1,(36)
and from (30) the accelerometer readings are
fb=Tb
n[agn+ (ωn
en ×+2ωn
ie×)vn].(37)
Once, the GT IMU readings (35) and (37) are obtained, noise
characteristics as described in Section III are added to create
our dataset.
REFERENCES
[1] J. Farrell, Aided navigation: GPS with high rate sensors. McGraw-Hill,
Inc., 2008, pp: 393-395.
[2] D. Wang, Y. Dong, Z. Li, Q. Li, and J. Wu, “Constrained mems-based
gnss/ins tightly coupled system with robust kalman filter for accurate
land vehicular navigation, IEEE Transactions on Instrumentation and
Measurement, vol. 69, no. 7, pp. 5138–5148, 2019.
[3] G. Wang, X. Xu, J. Wang, and Y. Zhu, “An enhanced ins/gnss tightly
coupled navigation system using time-differenced carrier phase measure-
ment,” IEEE Transactions on Instrumentation and Measurement, vol. 69,
no. 7, pp. 5208–5218, 2019.
[4] L. Paull, S. Saeedi, M. Seto, and H. Li, “AUV navigation and localiza-
tion: A review,” IEEE Journal of Oceanic Engineering, vol. 39, no. 1,
pp. 131–149, 2013.
[5] S. Gupte, P. I. T. Mohandas, and J. M. Conrad, A survey of quadrotor
unmanned aerial vehicles,” in 2012 Proceedings of IEEE Southeastcon.
IEEE, 2012, pp. 1–6.
[6] Q. Xu, X. Li, and C.-Y. Chan, “Enhancing localization accuracy of
mems-ins/gps/in-vehicle sensors integration during gps outages, IEEE
Transactions on Instrumentation and Measurement, vol. 67, no. 8, pp.
1966–1978, 2018.
[7] Y. Cui, S. Liu, J. Yao, and C. Gu, “Integrated positioning system of
unmanned automatic vehicle in coal mines,” IEEE Transactions on
Instrumentation and Measurement, vol. 70, pp. 1–13, 2021.
[8] Y. Bar-Shalom, X. R. Li, and T. Kirubarajan, Estimation with applica-
tions to tracking and navigation: theory algorithms and software. John
Wiley & Sons, 2004.
[9] X. Lyu, B. Hu, K. Li, and L. Chang, An adaptive and robust
UKF approach based on Gaussian process regression-aided variational
Bayesian,” IEEE Sensors Journal, vol. 21, no. 7, pp. 9500–9514, 2021.
[10] C. H. Kang, C. G. Park, and J. W. Song, “An adaptive complementary
kalman filter using fuzzy logic for a hybrid head tracker system,” IEEE
Transactions on Instrumentation and Measurement, vol. 65, no. 9, pp.
2163–2173, 2016.
[11] N. Davari and A. Gholami, “An asynchronous adaptive direct Kalman
filter algorithm to improve underwater navigation system performance,
IEEE Sensors Journal, vol. 17, no. 4, pp. 1061–1068, 2016.
[12] D.-J. Jwo and S.-H. Wang, “Adaptive fuzzy strong tracking extended
Kalman filtering for gps navigation, IEEE Sensors Journal, vol. 7, no. 5,
pp. 778–789, 2007.
[13] M. A. K. Jaradat and M. F. Abdel-Hafez, “Enhanced, delay dependent,
intelligent fusion for INS/GPS navigation system,” IEEE Sensors Jour-
nal, vol. 14, no. 5, pp. 1545–1554, 2014.
[14] X. Tong, Z. Li, G. Han, N. Liu, Y. Su, J. Ning, and F. Yang, Adaptive
ekf based on hmm recognizer for attitude estimation using mems marg
sensors,” IEEE Sensors Journal, vol. 18, no. 8, pp. 3299–3310, 2017.
[15] J. He, C. Sun, B. Zhang, and P. Wang, Adaptive error-state kalman filter
for attitude determination on a moving platform,” IEEE Transactions on
Instrumentation and Measurement, vol. 70, pp. 1–10, 2021.
[16] R. Mehra, “On the identification of variances and adaptive Kalman
filtering,” IEEE Transactions on Automatic Control, vol. 15, no. 2, pp.
175–184, 1970.
[17] L. Zhang, D. Sidoti, A. Bienkowski, K. R. Pattipati, Y. Bar-Shalom, and
D. L. Kleinman, “On the identification of noise covariances and adaptive
Kalman filtering: A new look at a 50 year-old problem, IEEE Access,
vol. 8, pp. 59 362–59 388, 2020.
[18] H. Yan, Q. Shan, and Y. Furukawa, “RIDI: Robust IMU double integra-
tion,” in Proceedings of the European Conference on Computer Vision
(ECCV), 2018, pp. 621–636.
[19] S. Herath, H. Yan, and Y. Furukawa, “Ronin: Robust neural inertial
navigation in the wild: Benchmark, evaluations, & new methods,”
in 2020 IEEE International Conference on Robotics and Automation
(ICRA). IEEE, 2020, pp. 3146–3152.
[20] C. Chen, P. Zhao, C. X. Lu, W. Wang, A. Markham, and N. Trigoni,
“Deep-learning-based pedestrian inertial navigation: Methods, data set,
and on-device inference,” IEEE Internet of Things Journal, vol. 7, no. 5,
pp. 4431–4441, 2020.
[21] O. Asraf, F. Shama, and I. Klein, “Pdrnet: A deep-learning pedestrian
dead reckoning framework, IEEE Sensors Journal, 2021.
PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 11
[22] A. Manos, T. Hazan, and I. Klein, “Walking direction estimation
using smartphone sensors: a deep network-based framework, IEEE
Transactions on Instrumentation and Measurement, 2022.
[23] M. Yona and I. Klein, “Compensating for partial doppler velocity log
outages by using deep-learning approaches,” in 2021 IEEE International
Symposium on Robotic and Sensors Environments (ROSE). IEEE, 2021,
pp. 1–5.
[24] D. Li, J. Xu, H. He, and M. Wu, “An underwater integrated navigation
algorithm to deal with dvl malfunctions based on deep learning,” IEEE
Access, vol. 9, pp. 82 010–82 020, 2021.
[25] M. Brossard and S. Bonnabel, “Learning wheel odometry and imu errors
for localization,” in 2019 International Conference on Robotics and
Automation (ICRA). IEEE, 2019, pp. 291–297.
[26] A. Shurin and I. Klein, “Quadnet: A hybrid framework for quadrotor
dead reckoning,” Sensors, vol. 22, no. 4, p. 1426, 2022.
[27] C. Shen, Y. Zhang, X. Guo, X. Chen, H. Cao, J. Tang, J. Li, and
J. Liu, “Seamless gps/inertial navigation system based on self-learning
square-root cubature kalman filter, IEEE Transactions on Industrial
Electronics, vol. 68, no. 1, pp. 499–508, 2020.
[28] B. Or and I. Klein, “Learning vehicle trajectory uncertainty,” arXiv
preprint arXiv:2206.04409, 2022.
[29] J. Liu and G. Guo, “Vehicle localization during gps outages with
extended kalman filter and deep learning,” IEEE Transactions on In-
strumentation and Measurement, vol. 70, pp. 1–10, 2021.
[30] M. Brossard, S. Bonnabel, and A. Barrau, “Denoising imu gyroscopes
with deep learning for open-loop attitude estimation,” IEEE Robotics
and Automation Letters, vol. 5, no. 3, pp. 4796–4803, 2020.
[31] P. D. Groves, Principles of GNSS, Inertial, and Multisensor Integrated
Navigation Systems 2nd ed. Boston, MA: Artech House, 2013.
[32] R. G. Brown and P. Y. Hwang, Introduction to random signals and
applied Kalman filtering. Wiley New York, Third Edtion, 1992.
[33] A. Shurin, A. Saraev, M. Yona, Y. Gutnik, S. Faber, A. Etzion, and
I. Klein, “The autonomous platforms inertial dataset (table 1),” IEEE
Access, vol. 10, pp. 10 191–10 201, 2022.
[34] A. L. Maas, A. Y. Hannun, A. Y. Ng et al., “Rectifier nonlinearities
improve neural network acoustic models, in Proc. icml, vol. 30, no. 1.
Citeseer, 2013, p. 3.
[35] L. Lu, Y. Shin, Y. Su, and G. E. Karniadakis, “Dying relu and initializa-
tion: Theory and numerical examples,” arXiv preprint arXiv:1903.06733,
2019.
[36] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization, arXiv
preprint arXiv:1607.06450, 2016.
[37] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,
arXiv preprint arXiv:1412.6980, 2014.
[38] DJI, “Matrice 300 RTK, Aircraft Specifications,” Website source:
https://www.dji.com/matrice-300/specs, 2021.
[39] A. Shurin, A. Saraev, M. Yona, Y. Gutnik, S. Faber, A. Etzion, and
I. Klein, “The autonomous platforms inertial dataset,” IEEE Access,
vol. 10, pp. 10 191–10 201, 2022.
[40] D.-J. Jwo, J.-H. Shih, C.-S. Hsu, and K.-L. Yu, “Development of a
strapdown inertial navigation system simulation platform, Journal of
Marine Science and Technology, vol. 22, no. 3, pp. 381–391, 2014.
Barak Or (Member, IEEE) received a B.Sc. degree
in aerospace engineering from the Technion–Israel
Institute of Technology, Haifa, Israel, a B.A. degree
(cum laude) in economics and management, and
an M.Sc. degree in aerospace engineering from the
Technion–Israel Institute of Technology in 2016 and
2018. He is currently pursuing a Ph.D. degree with
the University of Haifa, Haifa.
His research interests include navigation, deep learn-
ing, sensor fusion, and estimation theory.
Itzik Klein (Senior Member IEEE) received B.Sc.
and M.Sc. degrees in Aerospace Engineering from
the Technion-Israel Institute of Technology, Haifa,
Israel in 2004 and 2007, and a Ph.D. degree in Geo-
information Engineering from the Technion-Israel
Institute of Technology, in 2011. He is currently
an Assistant Professor, heading the Autonomous
Navigation and Sensor Fusion Lab, at the Hatter
Department of Marine Technologies, University of
Haifa. His research interests include data driven
based navigation, novel inertial navigation architec-
tures, autonomous underwater vehicles, sensor fusion, and estimation theory.
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Quadrotor usage is continuously increasing for both civilian and military applications such as surveillance, mapping, and deliveries. Commonly, quadrotors use an inertial navigation system combined with a global navigation satellite systems receiver for outdoor applications and a camera for indoor/outdoor applications. For various reasons, such as lighting conditions or satellite signal blocking, the quadrotor’s navigation solution depends only on the inertial navigation system solution. As a consequence, the navigation solution drifts in time due to errors and noises in the inertial sensor measurements. To handle such situations and bind the solution drift, the quadrotor dead reckoning (QDR) approach utilizes pedestrian dead reckoning principles. To that end, instead of flying the quadrotor in a straight line trajectory, it is flown in a periodic motion, in the vertical plane, to enable peak-to-peak (two local maximum points within the cycle) distance estimation. Although QDR manages to improve the pure inertial navigation solution, it has several shortcomings as it requires calibration before usage, provides only peak-to-peak distance, and does not provide the altitude of the quadrotor. To circumvent these issues, we propose QuadNet, a hybrid framework for quadrotor dead reckoning to estimate the quadrotor’s three-dimensional position vector at any user-defined time rate. As a hybrid approach, QuadNet uses both neural networks and model-based equations during its operation. QuadNet requires only the inertial sensor readings to provide the position vector. Experimental results with DJI’s Matrice 300 quadrotor are provided to show the benefits of using the proposed approach.
Article
Full-text available
One of the critical tasks required for fully autonomous functionality is the ability to achieve an accurate navigation solution; that is, to determine the platform position, velocity, and orientation. Various sensors, depending on the vehicle environment (air, sea, or land), are employed to achieve this goal. In parallel to the development of novel navigation and sensor fusion algorithms, machine-learning based algorithms are penetrating into the navigation and sensor fusion fields. An excellent example for this trend is pedestrian dead reckoning, used for indoor navigation, where both classical and machine learning approaches are used to improve the navigation accuracy. To facilitate machine learning algorithms’ derivation and validation for autonomous platforms, a huge quantity of recorded sensor data is needed. Unfortunately, in many situations, such datasets are not easy to collect or are not publicly available. To advance the development of accurate autonomous navigation, this paper presents the autonomous platforms inertial dataset. It contains inertial sensor raw data and corresponding ground truth trajectories. The dataset was collected using a variety of platforms including a quadrotor, two autonomous underwater vehicles, a land vehicle, a remote controlled electric car, and a boat. A total of 805.5 minutes of recordings were made using different types of inertial sensors, global navigation satellite system receivers, and Doppler velocity logs. After describing the sensors that were employed for the recordings, a detailed description of the conducted experiments is provided. The autonomous platform inertial dataset is available at: https://github.com/ansfl/Navigation-Data-Project/ .
Article
Full-text available
In underwater navigation systems, Global Navigation Satellite System (GNSS) information cannot be used for navigation. The mainstream method of autonomous underwater vehicles (AUV) underwater navigation system is Doppler Velocity Log (DVL) aided strapdown inertial navigation system (SINS). However, because the DVL is an instrument based on Doppler frequency shift to measure velocity, it is easily affected by the external environment. In a complex underwater environment, DVL output is easily polluted by outliers or even interrupted. In this paper, A new integrated navigation algorithm based on deep learning model is proposed to deal with DVL malfunctions. First, use RKF based on Mahalanobis distance algorithm to eliminate outliers, and then train the Nonlinear AutoRegressive with eXogenous input (NARX) model when DVL is available. When DVL is interrupted, use the NARX model to predict the output of DVL and continue integrated navigation. The proposed NARX-RKF scheme’s effectiveness verification was performed on the data set collected by the SINS/DVL ship-mounted experimental system. For comparison, different methods are also compared in the experiment. Experimental results show that NARX-RKF can effectively predict the output of DVL and is significantly better than other methods.
Article
Smartphone-based inertial and magnetic sensors can be the basis for pedestrian navigation, whenever external positioning signals are limited or unavailable. Such navigation solutions are typically accomplished by a practice known as pedestrian dead reckoning, wherein step length and heading angle are estimated to form the horizontal trajectory of the user. One of the main challenges in these methods is the unknown misalignment between user’s forward axis and device’s frame, which imposes great difficulty in finding the user’s heading. In this paper, the problem of estimating walking direction is addressed by a learning-based approach. Specifically, a novel deep network architecture is designed for extracting the motion vector in the device coordinates, using accelerometer measurements. It consists of temporal convolutions and multi-scale attention layers, and involves rotation-invariant property that is analytically derived. The network model is integrated with a geometric calculation, employing gravity and geomagnetic directions, to convert the motion vector into heading angle relative to north. Extensive experiments of natural walking activity were conducted by a single pedestrian, with smartphones placed freely in various pockets. These were used for training the proposed model, in a user-specific approach. Finally, the entire framework was evaluated on unseen data, comparing the deep network against a mechanistic baseline method. Using the proposed network, with merely 5500 parameters, the resulting heading errors had a median value of 9.8°, which was lower by 2.5° compared to the baseline method.
Article
Integration of microelectromechanical system-based inertial navigation system (MEMS-INS) and global positioning system (GPS) is a promising approach to vehicle localization. However, such a scheme may have poor performance during GPS outages and is less robust to measurement noises in changeable urban environments. In this article, we give an improved extended Kalman filter (IEKF) using an adaptation mechanism to eliminate the influence of noises in MEMS-INS and mitigate dependence on the process model. Especially, to guarantee accurate position estimation of the INS, a deep learning framework with multiple long short-term memory (multi-LSTM) modules is proposed to predict the increment of the vehicle position based on Gaussian mixture model (GMM) and Kullback-Leibler (KL) distance. The IEKF and the multi-LSTM are then combined together to optimize vehicle positioning accuracy during GPS outages in changeable urban environments. Numerical simulations and real-world experiments have demonstrated the effectiveness of the combined IEKF and multi-LSTM method, with the root-mean-square error (RMSE) of predicted position reduced by up to 93.9%. Or specifically, the RMSEs during GPS outages with durations 30, 60, and 120 s are 2.34, 2.69, and 3.08 m, respectively, which obviously outperform the existing method.
Article
This paper presents a loosely-coupled visual/inertial hybrid system for relative attitude determination on a moving platform. The inertial measurements are accomplished by two gyroscopes, of which the slave gyroscope removes redundant motion of the platform from the total motion sensed by the master gyroscope. To correct integration drift error and compensate for the gyroscope biases, we fuse the inertial and visual measurements fused together through the error-state Kalman filter (ESKF). Furthermore, an improved ESKF is proposed by utilizing the variational Bayesian (VB) inference to adaptively estimate the process and measurement noise covariance. To realize direct estimation of the process noise covariance, a latent variable is introduced, and inferred together with the noise covariances and error-state. The practical experimental setup is built up and experiments of two different motion trajectories are conducted. The ESKF, the cubature Kalman filter (CKF), the extended Kalman filter (EKF), the proposed filter and the other two adaptive filters are implemented for comparison to verify the effectiveness of the proposed filter. The results demonstrate the proposed filter show superiority in reducing both the root mean square error (RMSE) and maximum error (ME), and improving the filter consistency.
Article
Automatic positioning technology of underground vehicles has become the key technology for the intelligent development of coal mines. To improve the comprehensive positioning accuracy of unmanned automatic vehicles in coal mines, an 18-dimensional model of an odometer-aided inertial navigation system (INS) and a positioning model of an extended Kalman filter-based ultrawideband (UWB) are established based on the vehicle kinematics equation. A tight integrated state estimation model is proposed to restrain the long time drift of the INS and enhance the instability of the UWB system. A local positioning reference system based on an infrared motion capture system is established for the first time. Experimental tests show that the average positioning error of the proposed algorithm is 6.03 cm in the forward direction, and the root mean square error increased by 36.09% and 38.24% compared with those of the INS and UWB system, respectively. It is experimentally verified that the integrated positioning system achieves decimeter level positioning accuracy in a coal mine roadway environment.
Article
Pedestrian dead reckoning is a well-known approach for indoor navigation. There, the smartphone’s inertial sensors readings are used to determine the user position by utilizing empirical or bio-mechanical approaches and by direct integration. In this paper, we propose PDRNet, a deep-learning pedestrian dead reckoning framework, for user positioning. It includes a smartphone location recognition classification network followed by a change of heading and distance regression network. Experimental results using a publicly available dataset show that the proposed approach outperforms traditional approaches and other deep learning based ones.
Article
In this study, both adaptive and robust one is proposed, considering that the properties of a classic unscented Kalman filter (UKF) can be degraded severely by the outliers measured in the contamination distribution and the effects of time-varying noise. An adaptive and robust UKF approach on Gaussian process regression-assisted Variational Bayesian is proposed. The Variable Bayesian (VB) method is used to statistically approximate the time-varying noise due to the strong nonlinearity and inaccuracy of the system model. At the same time, Gaussian Process Regression (GPR) has the advantage of machine learning. The observation information trained by the GPR model performs real-time prediction of the sliding window. Based on the above two points, the output source of the measurement information is estimated to realize the robustness of the filter measurement update. When the noise is uncertain and there are outliers in the measurement information, simulation and SINS/GPS integrated navigation of actual tests were performed, respectively. The test outcomes indicate that the algorithm has better nonlinear estimation performance than the traditional method, the effectiveness of the filtering algorithm proposed is verified.