Available via license: CC BY 4.0

Content may be subject to copyright.

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 1

A Hybrid Model and Learning-Based

Adaptive Navigation Filter

Barak Or, Member, IEEE, and Itzik Klein, Senior Member, IEEE

Abstract—The fusion between an inertial navigation system

and global navigation satellite systems is regularly used in many

platforms such as drones, land vehicles, and marine vessels.

The fusion is commonly carried out in a model-based extended

Kalman ﬁlter framework. One of the critical parameters of the

ﬁlter is the process noise covariance. It is responsible for the

real-time solution accuracy, as it considers both vehicle dynamics

uncertainty and the inertial sensors quality. In most situations

the process noise is covariance assumed to be constant. Yet,

due to vehicle dynamics and sensor measurement variations

throughout the trajectory, the process noise covariance is subject

to change. To cope with such situations, several adaptive model-

based Kalman ﬁlters were suggested in the literature. In this

paper, we propose a hybrid model and learning-based adaptive

navigation ﬁlter. We rely on the model-based Kalman ﬁlter and

design a deep neural network model to tune the momentary

system noise covariance matrix, based only on the inertial sensor

readings. Once the process noise covariance is learned, it is

plugged into the well-established, model-based Kalman ﬁlter.

After deriving the proposed hybrid framework, ﬁeld experiment

results using a quadrotor are presented and a comparison to

model-based adaptive approaches is given. We show that the

proposed method obtained an improvement of 25% in the po-

sition error. Furthermore, the proposed hybrid learning method

can be used in any navigation ﬁlter and also in any relevant

estimation problem.

Index Terms—Adaptive Algorithm, Deep Learning, Global

Navigation Satellite System, Inertial Measurement Unit, Inertial

Navigation Systems, Kalman Filter, Machine Learning, Quad-

copter, Supervised Learning, Unmanned Autonomous Vehicles,

Vehicle Tracking.

I. INTRODUCTION

AUTONOMOUS vehicles, such as autonomous under-

water vehicles (AUV) or quadrotors, are commonly

equipped with an inertial navigation system (INS) and other

sensors [1] to provide real-time information about their posi-

tion, velocity, and orientation [2]–[7]. The INS has two types

of inertial sensors; namely, gyroscopes and accelerometers.

The former measures the angular velocity vector, and the latter

measures the speciﬁc force vector. The inertial sensors are

combined in an inertial measurement unit (IMU). Their use for

navigation is challenging due to their high noise levels, which

eventually accumulate, leading to a navigation solution drift

over time. To solve this problem, an external aiding sensor,

such as the global navigation satellite system (GNSS) receiver,

is regularly used in a a navigation ﬁlter [2], [3], [6]. Such

fusion is carried out using a nonlinear ﬁlter, where the error

state implementation of the extended Kalman ﬁlter (es-EKF)

is usually employed [1], [8]. In the ﬁlter, the process noise

Barak Or and Itzik Klein are with the Department of Marine Technologies,

Charney School of Marine Science, University of Haifa, Haifa, 3498838, Israel

(e-mail: barakorr@gmail.com, kitzik@univ@haifa.ac.il).

covariance matrix is based on the IMU measurements char-

acteristics and/or vehicle dynamics while the external aiding

measurements speciﬁcation determines the measurement noise

covariance matrix. Those two covariances matrices have a

major inﬂuence on the ﬁlter performance. A common practice

is to assume that both covariances are ﬁxed during the vehicle

operation. However, as the sensor characteristics are subject to

change during operation and physical constraints do not allow

capturing the vehicle dynamic optimally, those covariances

should vary over time as the amount of uncertainty varies and

is unknown. Thus, tuning the process or measurement noise

covariances optimally can lead to a signiﬁcant improvement

in the ﬁlter performance. [8].

To cope with this problem, several model-based attempts

have been made in the literature to develop optimal adaptive

ﬁlters [9]–[15]. The most common among them is based on

the ﬁlter innovation process [16], where a measure of the

new information is calculated at every iteration, leading to

an update to the process noise covariance matrix. Still, the

question of the optimal approach to tune the system noise

covariance matrix is considered open and is addressed in detail

in [17].

Recently, deep learning approaches were integrated in

model based pedestrian dead-reckoning (PDR) algorithms.

One of the initial works in the ﬁeld is the robust IMU double

integration, RIDI approach [18], where neural networks were

trained to regress linear velocities from inertial sensors to

constrain the accelerometer readings, although it was the ﬁrst

work in the ﬁeld, double integration is still needed and the

error accumulates rapidly. In [19], the device orientation, in

addition to the accelerometers and gyroscopes readings, was

also used as input to a deep learning architecture to regress the

user velocity in 2D. The velocity is then integrated to obtain

the user position. More pedestrian inertial navigation examples

are described by [20] where recent databases, methods and

real-time inferences can be found. Later, PDRNet [21], utilizes

a smartphone location recognition classiﬁcation network fol-

lowed by a change of heading and distance regression network

to estimate the used 2D position. One of the main challenges

in model or learning based PDR is the estimation of the user

walking direction as the device is not always aligned with the

user motion. To cope with that, in [22] a novel DNN structure

was designed for extracting the motion vector in the device

coordinates, using accelerometer readings.

In AUV navigation, an end-to-end deep learning approach

was proposed to regress missing Doppler velocity log (DVL)

beam measurements to provide the AUV velocity vector, only

when a single beam is missing. [23]. Later, [24], a deep

learning approach was used to address with a DVL failure

arXiv:2207.12082v1 [eess.SY] 14 Jun 2022

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 2

scenarios to predict the DVL output.

For land vehicle navigation a deep neural network (DNN)

model was presented to overcome the inaccurate dynamics

and observation models of wheel odometry for localization,

named RINS-W [25]. Their novel approach exploited DNNs

to identify patterns in wheeled vehicle motions using the IMU

sensor only. To overcome the navigation solution drift in pure

inertial quadrotor navigation, QuadNet, a hybrid framework to

estimate the quadrotor’s three-dimensional position vector at

any user-deﬁned time rate was proposed [26].

Focusing on the estimation process, a self-learning square

root-cubature Kalman ﬁlter was proposed [27]. This hybrid

navigation ﬁlter enhanced the navigation accuracy by provid-

ing observation continuously, even in GNSS denied environ-

ments, by learning transfer rules of internal signal in the ﬁlter.

In [28], recurrent neural networks are employed to learn the

vehicle’s geometrical and kinematic features to regress the

process noise value in a linear Kalman ﬁler framework. In

[29], DNN based multi models (triggered by trafﬁc conditions

classiﬁcation) together with an extended Kalman ﬁlter was

proposed to cope with GNSS outages. Recently, a DNN

model was proposed to solve the GNSS outages, mainly for

unmanned air vehicles (UAV). In [30], a convolutional neural

network model was used for noise-free gyro measurements in

open loop attitude estimation. They obtained state-of-the-art

navigation performance in terms of attitude estimation where

they compensated for gyro measurement errors as part of

a strapdown integration approach. This trend of integrating

learning approaches in various ﬁelds and applications also

raises the motivation to adopt such approaches in the described

problem.

In this paper, we propose a hybrid model and learning-

based adaptive navigation ﬁlter. We rely on the model-based

extended Kalman ﬁlter; yet, instead of using model-based

adaptive process noise tuning, we design a DNN to tune the

momentary system noise covariance matrix, based only on the

inertial sensor readings. Once, the process noise covariance

is learned, it is plugged into the well-established, model-

based Kalman ﬁlter. To that end, we simulated many vehicle

trajectories based on six baseline trajectories to create a rich

dataset. This dataset contains rich dynamic scenarios with

many motion patterns and different IMU noise covariances.

This rich dataset was created to enable robustness to motion

dynamics and IMU characteristics. Based on this dataset, we

adopt a supervised learning (SL) approach to regress the

process noise covariance.

The main contributions of this paper are:

1) Derivation of an adaptive hybrid learning algorithm to

momentary determine the process noise covariance matrix

as a function of the IMU measurements. That is, instead

of using model-based approaches (as been done in the

last 50 years), to the best of our knowledge, we are the

ﬁrst to propose an adaptive hybrid learning algorithm.

2) Online integration of the proposed hybrid learning al-

gorithm with es-EKF implementation of the navigation

ﬁlter. In that manner, the proposed approach can be used

with any external sensor aiding the INS.

3) The proposed approach principles can be further applied

to any estimation problem that requires adaptive tuning

of the process noise covariance.

The advantages of the proposed approach lies in its hybrid

fusion of the well celebrated EKF and applying learning

approaches on the EKF soft spot, the adaptive determination

of the process noise covariance. By using learning approaches

we leverage from their ability to generalization of intrinsic

properties appearing in sequential datasets and, therefore,

better cope with varying conditions affecting the process noise

value.

After deriving and validating the proposed framework using

the stimulative dataset, ﬁeld experiments using a quadrotor

are done to evaluate the robustness and performance of the

proposed approach testing phase and compare it to other adap-

tive model-based approaches. The recordings from the ﬁeld

experiment are used as a test dataset and also to examine our

network capability for generalization. In our setup, we used

GNSS position measurements to update the INS; however, the

proposed approach is suitable for any aiding sensor and for

any platform.

The rest of the paper is organized as follows: Section II

presents the problem formulation for the INS/GNSS fusion in

an es-EKF implementation with constant and adaptive process

noise covariance. Section III presents our proposed hybrid

adaptive EKF. Section IV gives the results, and Section V

presents the conclusions of this study.

II. ADA PT IV E NAVIG ATIO N FILTER

The nonlinear nature of the INS equations requires a nonlin-

ear ﬁlter. The most common ﬁlter for fusing INS with external

aiding sensors is the EKF [1], and in particular, with a 15 error

states implementation:

δx=δpnδvnδεnbabgT∈R15×1,(1)

where δpn∈R3×1is the position vector error states expressed

in the navigation frame, δvn∈R3×1is the velocity vector

error states expressed in the navigation frame, δεn∈R3×1is

the misalignment vector expressed in the navigation frame,

ba∈R3×1is the accelerometer bias residuals vector ex-

pressed in the body frame, and bg∈R3×1is the gyro bias

residuals vector expressed in the body frame. The navigation

frame is denoted by nand the body frame is denoted by b. The

navigation frame center is located at the body center of mass,

where the x-axis points to the geodetic north, z-axis points

down parallel to local vertical, and y-axis completes a right-

handed orthogonal frame. The body frame center is located

at the center of mass, x-axis is parallel to the longitudinal

axis of symmetry of the vehicle pointing forwards, the y-axis

points right, and the z-axis points down such that it forms

a right-handed orthogonal frame. The linearized, error-state,

continuous-time model is

δ˙x =Fδx+Gδw,(2)

where F∈R15×15 is the system matrix, G∈R15×12 is

the shaping matrix, and δw=wawgwab wgb T∈

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 3

R12×1is the system noise vector consisting of the accelerom-

eter, gyro, and their biases random walk noises, respectively

[8]. The system matrix, F, is given by

F=

Fpp Fpv Fpε 03×303×3

Fvp Fvv Fvε −Tn

b03×3

Fεp Fεv Fεε 03×3Tn

b

03×303×303×303×303×3

03×303×303×303×303×3

,(3)

where Tn

bis the transformation matrix between body and

navigation frame and Fij ∈R3×3can be found explicitly

in the literature (see [1], [31], [32]). The shaping matrix is

given by

G=

Tn

b03×303×303×3

03×3−Tn

b03×303×3

03×303×303×303×3

03×303×3I3×303×3

03×303×303×3I3×3

,(4)

where I3×3is an identity matrix.

The corresponding discrete version of the navigation model

(for small step sizes), as given in (2), is

δxk+1 =Φkδxk+Gkδwk.(5)

The transition matrix, Φk, is deﬁned by a ﬁrst order approxi-

mation as

Φk

∆

=I+F∆t, (6)

where kis a time index, ∆tis the time step size, Fis the

system matrix (obtained using (3) with the estimated state

vector at time k), and δwkis a zero mean white Gaussian

noise. The discretized process noise is given by

Qd

k=GQcGT∆t, (7)

where Gis deﬁned in (4) (using the current estimated state

vector at time k), and Qcis the continuous process noise

matrix.

The discrete es-EKF, as described here, is used in the fusion

process between the INS and the external measurements.

Equations (8)−(12) summarize the navigation ﬁlter equations

[1], [31]:

δˆx−

k=0(8)

P−

k=Φk−1Pk−1Φk−1T+Qd

k−1(9)

Kk=P−

kHkThHkP−

kHkT+Rd

ki−1

(10)

δˆxk=Kkδzk(11)

Pk= [I−KkHk]P−

k,(12)

where δˆx−

kis the prior estimate of the error state and δˆxk

is the posterior estimate, with their corresponding covariance

matrices P−

k(9) and Pk(12), respectively, Kk(10) is the

Kalman gain, δzkis the kmeasurement residual, Hkis

the measurement matrix, and Rkis the measurement noise

covariance, assumed known and constant.

A. Measurement Model

The GNSS position measurements are available in a con-

stant and lower frequency then that of the IMU. After pro-

cessing, the GNSS receiver outputs the vehicle position vector

in the navigation frame, where it is used as an update in

the navigation ﬁlter. Hence, the corresponding time-invariant

GNSS measurement matrix is given by

HGNS S =I3×303×12 ∈R3×15.(13)

The corresponding measurement residual is given by

δzGNS S ,j =HGN SS δxj+ςGN S S ,j ,(14)

where ςGNS S ,j ∼ N 0,RdGN SS ∈R3×1is an additive,

discretized, zero mean white Gaussian noise. It is assumed

that ςjand δwjare uncorrelated.

B. Model Based-Adaptive Noise Covariance

The es-EKF minimizes the tracking error without changing

Qmatrix online; that is, the process noise covariance is

constant throughout the operation. As shown in the literature,

[17] and the references therein, tuning Qonline greatly

improves the navigation ﬁlter performance .

For simplicity, we assume Qd

kis a diagonal matrix, thus no

correlation exists between the noise terms.

The optimal diagonal Qmatrix at time step kis deﬁned by

Q∗

k

∆

= arg min

Qk∈Q

ˆxk(Qk)−xGT

k

2

2,(15)

where Qis the admissible set of Qkat time index k,xGT

kis

the ground truth (GT) state at time index k, and k·k2is the

second Euclidean norm.

The most common approach to estimate Qin an adaptive EKF

framework was suggested in [16], [17], and is based on the

innovation matrix for a window of size ξ:

Ck

∆

=1

ξ

k

X

i=k−ξ+1

νiνiT,(16)

where Ckis the innovation matrix and the innovation vector

is deﬁned by

νk

∆

=zk−Hˆx−

k.(17)

The innovation matrix, (16), is used together with the Kalman

gain, K, to adapt Q, as follows:

ˆ

Qk=KkCkKT

k.(18)

III. A HYBRID ADAP TI VE NAVI GATI ON FI LTER

We rely on the well established, model-based EKF, yet

instead of a model-based adaptive approach, a data-driven

approach is used to regress the momentary process noise

covariance, and, as a result, creates a hybrid adaptive EKF.

To this end, the navigation ﬁlter equations (8)-(12) are used,

but instead of using a constant process noise (7) or a model-

based adaptive process noise (18), we derive a data-driven ap-

proach to estimate the time-varying continuous process noise

covariance matrix Qc. Our proposed hybrid adaptive EKF

framework, suitable for any aided INS scenario, is presented

in Figure 1.

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 4

Fig. 1. A hybrid adaptive navigation ﬁlter framework with online tuning of

the process noise covariance matrix. .

A. Data-Driven Based Adaptive Noise Covariance

It is assumed that the continuous process noise covariance

matrix, Qc

k, at time k, is a diagonal matrix with the following

structure:

Qc

k=

diagnqx∗

f, qy∗

f, qz∗

f, qx∗

ω, qy∗

ω, qz∗

ω, εI1×6ok∈R12×12,(19)

where the variance for each of the accelerometer axes is given

by qi∗

fand for the gyroscope by qi∗

ω(i∈x, y, z). The biases

are modeled by random walk processes, and their variances are

set to ε= 0.001. By taking this approach, our goal is to ease a

machine learning ML algorithm for accurately regressing the

six unknown variances in (19) and thereby claim that these

are the dominant variances needed in an adaptive framework.

We deﬁne a general one-dimensional series of length Nof

the inertial sensor (accelerometer or gyroscope) readings in a

single axis by

Sk={si}k−1

i=k−N.(20)

The discussed optimization problem is to ﬁnd q∗

ksuch that

the position error is minimized. By doing that for all six IMU

channels, the es-EKF considers an optimal system noise co-

variance matrix and provides improved tracking performance.

The power of ML increases the ability to solve many difﬁcult

and non-conventional tasks. To determine Q∗

kin (15), the

problem is formulated as a SL problem. Formally, we search

for a model to relate an instance space, X, and a label space,

Y. We assume that there exists a target function, F, such that

Y=F(X).

Thus, the SL task for adaptive determination of the process

noise covariance is to ﬁnd F, given a ﬁnite set of labeled

examples, inertial sensor readings, and corresponding process

noise variance values:

{Sk, q∗

k}M

k=1 .(21)

The SL approach aims to ﬁnd a function ˜

Fthat best estimates

F. To that end, for the training process, a loss function, l, is

deﬁned to quantify the quality of ˜

Fwith respect to F. The

loss function is given by

LY,ˆ

Y(X)∆

=1

M

M

X

m=1

l(y, ˆy)m,(22)

where Mis the number of examples, and mis the example

index. Minimizing Lin a training/test procedure leads to the

target function. In our problem, the loss function is given by

lm

∆

= (q∗

m−ˆqm)2,(23)

where ˆqmis the estimated term obtained by the learning model

during the training process.

In this work, the train dataset used to ﬁnd the function ˜

Fgiven

the labeled examples (21), as described in the next section.

B. Dataset Generation

There are several existing inertial datasets as were recently

summarized in [33]. Yet, none of those datasets ﬁts our

problem as they do not provide enough IMU readings with

different noise characteristic. As a consequence, we generated

are own dataset using a simulation (detailed in Appendix B).

To generate the dataset six different baseline trajectories

were simulated as presented in Figure 2(a). The richness

of the trajectories, as a result of their diversity, allows the

establishment of a model to cope with unseen trajectories.

Each baseline trajectory was created by generating ideal IMU

readings for a period of 400s in a sampling rate of 100Hz,

resulting in a sequence of 6×40,000 samples for each baseline

trajectory as shown in Figure 2.

Our task is to estimate the system noise covariance matrix, or

speciﬁcally the noise variance of each IMU channel. Hence,

we divided up each of the six IMU channels and corrupted

their perfect data with additive white Gaussian noise with

variance in the range of q∈[0.001,0.025], with 15 different

values inside this interval, as shown in Figure 2(b).

Hence, each baseline trajectory has 15 series of 6×40,000

noisy inertial samples, as shown in Figure 2(b). The justiﬁ-

cation to include a simple noise model lies in the momentary

IMU measurement noise covariance sequence. A short time

window for the IMU measurements is considered and allows

characterizing the noise with its variance only. Next, the series

length Nis chosen, and a corresponding labeled database is

generated as shown in Figure 2(c). Lastly, we choose N= 200

samples and create batches corresponding to two seconds each

with a total of 6×200×15. Then, these batches are randomly

divided into train and test datasets in such a manner that all

baseline trajectories are included in the train and the test sets

in a ratio of 80:20, as described in Figure 2(d).

C. Architecture design

Several architectures were designed and evaluated. They

include recurrent DNN, and, in particular, the long short-term

memory (LSTM) architectures for time series data. However,

in our problem they did not perform well and failed in

generalization of the noise variance property. One of the

architecture we examined included a bi-directional LSTM

layer followed by two linear layers. The training failed as

the temporal information wasn’t relevant for this task. On

the other hand, an architecture with only convolution layers

achieved satisfactory results, where it minimized the test loss

very fast and obtained a low position root mean squared

error (PRMSE) (the metric is deﬁned in (25)).This result are

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 5

Fig. 2. Dataset generation and pre-processing phase: (a) six different simulated baseline trajectories. Each baseline trajectory was created by tuning the IMU

signals a period of 400[s]at a frequency of 100[Hz], resulting in a series of 6×40,000 samples for each baseline trajectory, obtained in phase (b). Example

generation (c): given series length, N, examples are created and stored with their label. (d) split the database into train set and test set with a ratio of 80:20.

Fig. 3. Our six baseline trajectories.

due to the spatial information the noisy IMU signals have.

This information structure was easily captured by the spatial

convolutional operates in the convolution layers. As the goal

of this paper is to lay the foundations for a hybrid, adaptive

navigation ﬁlter approach, we describe here only the DNN

architecture as presented in Figure 4. Our DNN model has the

following layers:

1) Linear layer: This is the simpliﬁed layer structure that

applies a linear transformation to the incoming data from

the previous layer. The input features are received in the

form of a ﬂattened 1D vector and are multiplied by the

weighting matrix.

2) Conv1D layer: A convolutional, one-dimensional

(Conv1D) layer creates a convolution kernel that is

convolved with the layer input over a single spatial

dimension to produce a vector of outputs. During the

training phase the DNN learns the optimal weights

of the kernels. In our DNN model the input layer is

followed by a chain of three Conv1D layers. There are

ﬁve kernels for the ﬁrst layer and three kernels for the

second and third layers. The kernel size decreases as

the inputs go deep: the ﬁrst Conv1D layer has twenty

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 6

kernels, the second Conv1D layer has ten kernels, and

the third Conv1D layer has ﬁve kernels.

3) Global average pooling layer: Pooling layers help with

better generalization capability as they perform down

sampling feature map. In this way, the robustness of the

DNN grows. For example, better handling with changes

in the position of the features inside the 1D input vector.

For the DNN architecture we selected the global average

pooling method, which calculates the average for each

input channel and ﬂattens the data so it passes from

Conv1D through the linear layers.

4) Leaky ReLU: Leaky rectiﬁed linear unit [34] is a non-

linear activation function obtaining a value αwith the

following output:

f(α) = α α > 0

0.01α α ≤0.(24)

The main advantage of the leaky ReLU over the classical

ReLU (0 for α≤0) is the small positive gradient when

the unit is not active, which deals better with the dying

ReLU problem [35]. The leaky ReLU functions were

combined after each layer in the DNN model.

5) Layer normalization: One of the challenges of DNN

is that the gradients with respect to the weights in one

layer are highly dependent on the outputs of the neurons

in the previous layer, especially if these outputs change

in a highly correlated way [36]. Batch normalization, and

in particular layer normalization, was proposed to reduce

such an undesirable covariate shift. The layer normaliza-

tion was added after every Conv1D layer (together with

the leaky ReLU layer).

The parameters of the above layers are as follow: a series of

200×1samples is inserted to a one-dimensional convolutional

layer (Conv1D) with a kernel size equal to 20 with ﬁve ﬁlters.

Then, a leaky ReLU activation function and normalization

layer are applied, adding nonlinearity for better generalization

capability of the DNN. We duplicated this structure twice

more with three ﬁlters each and smaller kernels: the second

with a kernel size equals to ten and the third with a kernel

size equals to ﬁve. The output is inserted to a global average

pooling layer followed by four linear layers, with leaky ReLU

activation functions between them, with 100,80,50 and 20

weights, retrospectively. Finally, the DNN outputs the process

noise covariance, q∗.

D. Hybrid Learning Approach

In this work, GNSS position updates are employed to

aid the INS and demonstrate our hybrid learning approach.

Applying the suggested adaptive tuning approach in online

setting involves integrating the INS/GNSS with the regressor

as presented in Figure 5. Algorithm 1 describes the INS/GNSS

with process noise covariance learning in a real time setup.

The IMU signals are inserted both into the INS/GNSS ﬁlter

(8) −(12) and the DNN model. The regressed continuous

process noise covariance is plugged into (7) to obtain the

discrete one, which in turn is substituted into (9).

Algorithm 1 Hybrid adaptive ﬁlter applied to INS/GNSS

Input: ωib,fb,vAiding ,∆t0,∆τ, T, tuning Rate

Output: vn, εn

Initialization :vn

0, εn

0

LOOP Process

1: for t= 0 to Tdo

2: obtain ωib,fb

3: solve navigation equations (3)

4: if (mod(t, ∆τ)=0) then

5: obtain pGNS S

6: update navigation state using the es-EKF (8)-(14)

7: end if

8: Calculate DNN model and predict Qc

k+1.

9: if mod(t, tuningRate) = 0 then

10: Qc

k+1 =˜

FNN (Sk)

11: end if

12: end for

IV. ANA LYSI S AN D RES ULTS

We employ the following two metrics to evaluate the

position accuracy of the proposed approach:

•Position root mean squared error (2nd norm) for all three

axes:

P RM SE =v

u

u

t

1

c

c

X

k=1 X

j∈{x,y,z}

δˆp2

jk .(25)

•Position mean absolute error:

P M AE =1

c

c

X

k=1 X

j∈{x,y,z}

|δˆpjk |.(26)

where c is the number of samples, kis a running index for the

samples, jis the running index for the Cartesian coordinate

system, and δˆpjk is the position error term.

A. Train Dataset Analysis

The proposed DNN architecture, presented in Section III-C,

was trained on the dataset described in Section III-B. After 30

epochs it achieved an RMSE of 0.0032 on the test set, show-

ing the proposed DNN generalization capability and thereby

establishing the trained DNN regressor model. The training

process was done in mini-batches of 500 examples each. The

ADAM optimizer [37] was used with an initial learning rate

of 0.001 and a learn-rate-schedule with a drop factor of 0.1

after 20 epochs. In our working environment (Intel i7-6700HQ

CPU@2.6GHz 16GB RAM with MATLAB), the training time

for input length of 200 samples elapsed about three hours.

The averaged inference time is 0.02[s]). Figure 6 presents

the performance for N= 200 samples where the red line

is the desired one, representing when the GT value is equal

to the DNN predicted value. The gray points represent the

performance of the test set with the trained DNN model. The

mean values of the test set lies near the red line (blue points)

for most cases.

The INS/GNSS fusion is mostly performed under real-

time conditions, where latency in the position computation

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 7

Fig. 4. DNN architecture: Three Conv1D layers are followed by four linear layers and a global average pooling layer. Leaky ReLU activation functions are

used after every layer to add nonlinearity and achieve better generalization capability.

Fig. 5. Our adaptive hybrid adaptive ﬁlter applied to the INS/GNSS fusion

process to tune the process noise covariance matrix, ˆ

Qk. Both IMU and

GNSS measurements are plugged into the INS/GNSS ﬁlter, while the six

IMU channels are also inserted into the DNN for predicting the process noise

covariance matrix.

Fig. 6. GT values versus predicted DNN values on the test set. All values

are presented by gray crosses. The blue mean points represent the mean for

each value, and the dashed red line (true value=predict value) represents the

DNN generalization capability.

might degrade the performance. Thus, the inﬂuence of the

IMU sequence length input was examined. The system noise

covariance matrix was learned, based on series of length N. As

N >> 1, the probability of learning the correct terms grows,

since the DNN can capture the signal intrinsic properties easier

using more data; however, the latency grows. Considering this

trade-off, we trained the chosen architecture with various N

values and calculated the RMSE as shown in Figure 7. As

expected, N= 400 obtained the minimum RMSE, yet for the

rest of the analysis we chose N= 200 (similar RMSE) to

receive the regression result as it is given in a shorter time

period. For example, in the stimulative train and test dataset

the sampling rate is 100Hz, thus working with N= 200 gives

the regression result every two seconds instead of working

with four seconds (N= 400).

Fig. 7. DNN RMSE vs. input length (N).

B. Field experiment

A ﬁeld experiment with the DJI Matrice 300 [38] was

performed (Figure 8). The trajectory is shown in Figure 9.

An alike ﬁgure-eight ﬂight trajectory was made with some

additional approximated straight segments. The experiment

parameters, including the GNSS RTK GT values and IMU,

are given in Table I. The quadrotor dataset was published

in [39] and is publicly available at https://github.com/ansﬂ/

Navigation-Data-Project.

Equations (2)-(12) were used as a model-based EKF with

a constant process noise covariance matrix. Three different

cases of the model-based EKF with constant process noise

covariance matrix (MB-EKF-CQ) were examined:

•MB-EKF-CQ1: q1:3 = 0.002, q4:6 = 0.02.

•MB-EKF-CQ2: q1:3 = 0.001, q4:6 = 0.01.

•MB-EKF-CQ3: q1:3 = 0.002, q4:6 = 0.01.

In addition, using (16)-(18), three different (window size

length) cases of model-based EKF with the adaptive process

noise covariance matrix (MB-EKF-AQ) were examined:

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 8

Fig. 8. DJI matrice 300 in our ﬁeld experiment.

Fig. 9. The trajectory of the quadrotor used to evaluate the proposed approach.

•MB-EKF-AQ1: ξ= 1.

•MB-EKF-AQ2: ξ= 3.

•MB-EKF-AQ3: ξ= 5.

All six approaches were compared to our proposed hybrid

adaptive EKF (HB-AEKF) using the trained network, as

described in the previous section. Results in terms of PMAE

and PRMSE are presented in Table 2. The MB-EKF-AQ

model-based adaptive ﬁlters obtained better performance than

the MB-EKF-CQ constant process noise ﬁlters. In particular,

MB-EKF-CQ3 achieved the best performance for constant

process noise with a PMAE of 2.2[m]and a PRMSE of

1.6[m], where MB-EKF-AQ3 manged to improve them by

18% and 19%, respectively. Our proposed hybrid adaptive

approach, HB-AEKF, with an input length of 200 samples,

obtained the best performance, with 1.7[m]PMAE and

1.2[m]PRMSE improving MB-EKF-AQ3 by 5.5% and 7.7%,

respectively.

The qparameters time evaluation were obtained using

our HB-AEKF approach—are shown in Figure 10 for both

accelerometers and gyroscopes. Their behavior shows ﬂuctua-

tions in the ﬁrst 25 seconds and then steady-state behavior.

This behavior can be attributed to the convergence of the

Kalman ﬁlter (8)-(12) once it obtains enough data and cap-

tures the IMU noise statistic.

TABLE I

EXP ERI ME NT PAR AME TER S

Description Symbol Value

GNSS noise (var) - horizon R11, R22 0.01[m]2

GNSS noise (var) - vertical R33 0.02[m]2

GNSS step size ∆τ0.2[s]

IMU step size ∆t0.001[s]

Accelerometer noise (MPU-9250) q∗

1:3 0.022[m/s]2

Gyroscope rate noise (MPU-9250) q∗

4:6 0.0022[rad/s]2

Experiment duration T60[s]

Initial velocity vn

0[0,0,0]T[m/s]

Initial position pn

0[32.10,34.80,0]T

TABLE II

EXP ERI ME NT RE SULT S SHO WIN G PMAE AN D PRMSE PERFORMANCE

Approach PMAE[m]PRMSE[m]

HB-AEKF (ours) 1.7 1.2

MB-EKF-AQ1 2.3 1.6

MB-EKF-AQ2 2.0 1.4

MB-EKF-AQ3 1.8 1.3

MB-EKF-CQ1 2.4 1.8

MB-EKF-CQ2 2.4 1.7

MB-EKF-CQ3 2.2 1.6

Figure 11 summarizes the positioning performance by provid-

Fig. 10. The process noise parameters as a function of time as obtained

using our HB-AEKF approach. The upper plot shows the accelerometer noise

variance for each axis as detected by the DNN model, and the lower plot

shows the same for the gyroscope.

ing two graphs of the cumulative density function (CDF); one

representing PRMSE and the other the PMAE. In those plots,

all three MB-EKF-AQ and three MB-EKF-CQ approaches are

presented as well as our proposed HB-AEKF hybrid approach

(black lines). Both graphs show clearly that the entire error is

lower for HB-AEKF than all other approaches. The PRMSE of

the MB-AEKF is lower than 1.7[m]and the PMAE lower than

2.8[m]for the entire scenario. In comparison with the MB-

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 9

EKF-AQ1 approach, the maximum PRMSE and the maximum

PMAE might grow too much: 3.7[m]and 4.0[m], respectively.

Fig. 11. PRMSE and PMAE CDF graphs.

V. CONCLUSIONS

The proper choice for the system noise covariance matrix

is critical for accurate INS/GNSS fusion. In this paper a

hybrid learning and model based approach was suggested

in an adaptive ﬁlter framework. To that end, a novel DNN-

based approach to learn and online tune the system noise

covariance was developed. The input to the DNN is only the

IMU readings. This learning approach was then combined with

the model based es-EKF, resulting in a hybrid adaptive ﬁlter.

The DNN model has been trained only once on a simulated

database consisting of six different baseline trajectories. Anal-

ysis on this dataset showed a trade-off between accuracy and

the regression solution latency, where as the latency increases

the accuracy decreases. Relying on this analysis, an input

length of 200 samples was chosen for the analysis.

To validate the performance of the proposed approach, it

was compared to six model-based approaches representing

both constant and adaptive process noise covariance selection.

Quadrotor ﬁeld experiments were conducted and used as an

additional test dataset on the trained network. We obtained

state-of-the-art results for each noise value compared to clas-

sical adaptive KF approaches; a PRMSE lower than 1.2[m]

and a PMAE lower than 1.7[m]were obtained using our

suggested approach, compared to a PRMSE of 1.6[m]and

a PMAE of a 2.3[m]that were obtained using the innovation-

based adaptive approach and classical tuning. Results in a

total PRMSE improvement of 24% and PMAE improvement

of 27%.

This improvement also demonstrates the robustness and gen-

eralization properties of the proposed DNN architecture, en-

abling it to cope with unseen data from different IMU sensors.

In addition, and together with the train dataset results, it proves

our hypothesis of regressing only the ﬁrst six elements in the

diagonal of the continuous process noise covariance.

Although demonstrated for quadrotor INS/GNSS fusion, the

proposed approach can be elaborated for any external sensor

aiding the INS and for any type of platform.

APPENDIX

A. Abbreviations

TABLE III

ABBREVIATION AND DESCRIPTION

Abbreviation Description

AUV autonomous underwater vehicle

CDF cumulative density function

Conv1D one-dimensional convolution

DNN deep neural network

DVL Doppler velocity log

EKF extended Kalman ﬁlter

es error state

GNSS global navigation satellite system

GT ground truth

HB-AEKF Hybrid Based Adaptive EKF

INS inertial navigation system

LSTM long short-term memory (LSTM)

MB-EKF-AQ Model Based EKF Adaptive Q

MB-EKF-CQ Model Based EKF Constant Q

ML machine learning

NED North-East-Down

PDR pedestrian dead-reckoning

PMAE position mean absolute error

PRMSE position RMSE

RMSE root mean squared error

SL supervised learning

UAV unmanned air vehicles

B. Dataset Generation

1) INS Equations of Motion: The INS equations of motion

include the rate of change of the position, velocity, and the

transformation between the navigation and body frame, as

shown in Fig.12.

The position vector is given by

pn=φλhT∈R3×1,(27)

where φis the latitude, λis the longitude, and his the altitude.

The velocity vector is Earth referenced and expressed in the

North-East-Down (NED) coordinate system:

vn=vNvEvDT∈R3×1,(28)

where vN, vE, vDdenote the velocity vector components in

north, east, and down directions, respectively. The rate of

change of the position is given by [1]

˙pn=

˙

φ

˙

λ

˙

h

=

vN

RM+h

vE

cos(φ)(RN+h)

−vD

,(29)

where RMand RNare the meridian radius and the normal

radius of curvature, respectively. The rate of change of the

velocity vector is given by [1]

˙vn=Tn

bfb+gn−([ωn

en×] + 2 [ωn

ie×]) vn,(30)

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 10

where Tn

b∈R3×3is the transformation matrix from body

frame to the navigation frame. fb∈R3×1is the accelerometers

vector expressed in the body frame, gn∈R3×1is the gravity

vector expressed in the navigation frame. ωn

en is the angular

velocity vector between the earth centered earth ﬁxed (ECEF)

frame and the navigation frame. The angular velocity vector

between ECEF and the inertial frame is given by ωn

ie and the

rate of change of the transformation matrix is given by [1]

˙

Tn

b=Tn

bωb

ib×−ωb

in×,(31)

where ωb

ib =p q r T∈R3×1is the angular velocity

vector as obtained by the gyroscope and ωb

in is the angular

velocity vector between the navigation frame and the inertial

frame expressed in the body frame. The angular velocity

between the navigation frame and the inertial frame expressed

in the navigation frame is given by ωn

in. The alignment

between body frame and navigation frame can be obtained

from Tn

b, as follows

ε=

ϕ

θ

ψ

=

atan2Tb

n31,Tb

n32

arccos Tb

n33

−atan2Tb

n13,Tb

n23

∈R3×1,(32)

where ϕis the roll angle, θis the pitch angle, and ψis the

yaw angle. These three angles are called Euler angles.

Fig. 12. Autonomous underwater vehicle and quadcopter with their body axes

and angular rates.

2) IMU readings: The GT trajectories were created by

tuning vehicle angular rates, P, and vehicle accelerations, a,

as follows [40]

P=˙ϕ˙

θ˙

ψT∈R3×1,(33)

and,

a=axayazT∈R3×1.(34)

The transformation between the desired motion and the INS

framework for the gyroscope readings is given by [40]

ωb

ib =Tb

n(ωn

en +ωn

ie) + ωb

ib (35)

where

ωb

ib =

1 0 −sin θ

0 cos ϕsin ϕcos θ

0−sin ϕcos ϕcos θ

P ∈ R3×1,(36)

and from (30) the accelerometer readings are

fb=Tb

n[a−gn+ (ωn

en ×+2ωn

ie×)vn].(37)

Once, the GT IMU readings (35) and (37) are obtained, noise

characteristics as described in Section III are added to create

our dataset.

REFERENCES

[1] J. Farrell, Aided navigation: GPS with high rate sensors. McGraw-Hill,

Inc., 2008, pp: 393-395.

[2] D. Wang, Y. Dong, Z. Li, Q. Li, and J. Wu, “Constrained mems-based

gnss/ins tightly coupled system with robust kalman ﬁlter for accurate

land vehicular navigation,” IEEE Transactions on Instrumentation and

Measurement, vol. 69, no. 7, pp. 5138–5148, 2019.

[3] G. Wang, X. Xu, J. Wang, and Y. Zhu, “An enhanced ins/gnss tightly

coupled navigation system using time-differenced carrier phase measure-

ment,” IEEE Transactions on Instrumentation and Measurement, vol. 69,

no. 7, pp. 5208–5218, 2019.

[4] L. Paull, S. Saeedi, M. Seto, and H. Li, “AUV navigation and localiza-

tion: A review,” IEEE Journal of Oceanic Engineering, vol. 39, no. 1,

pp. 131–149, 2013.

[5] S. Gupte, P. I. T. Mohandas, and J. M. Conrad, “A survey of quadrotor

unmanned aerial vehicles,” in 2012 Proceedings of IEEE Southeastcon.

IEEE, 2012, pp. 1–6.

[6] Q. Xu, X. Li, and C.-Y. Chan, “Enhancing localization accuracy of

mems-ins/gps/in-vehicle sensors integration during gps outages,” IEEE

Transactions on Instrumentation and Measurement, vol. 67, no. 8, pp.

1966–1978, 2018.

[7] Y. Cui, S. Liu, J. Yao, and C. Gu, “Integrated positioning system of

unmanned automatic vehicle in coal mines,” IEEE Transactions on

Instrumentation and Measurement, vol. 70, pp. 1–13, 2021.

[8] Y. Bar-Shalom, X. R. Li, and T. Kirubarajan, Estimation with applica-

tions to tracking and navigation: theory algorithms and software. John

Wiley & Sons, 2004.

[9] X. Lyu, B. Hu, K. Li, and L. Chang, “An adaptive and robust

UKF approach based on Gaussian process regression-aided variational

Bayesian,” IEEE Sensors Journal, vol. 21, no. 7, pp. 9500–9514, 2021.

[10] C. H. Kang, C. G. Park, and J. W. Song, “An adaptive complementary

kalman ﬁlter using fuzzy logic for a hybrid head tracker system,” IEEE

Transactions on Instrumentation and Measurement, vol. 65, no. 9, pp.

2163–2173, 2016.

[11] N. Davari and A. Gholami, “An asynchronous adaptive direct Kalman

ﬁlter algorithm to improve underwater navigation system performance,”

IEEE Sensors Journal, vol. 17, no. 4, pp. 1061–1068, 2016.

[12] D.-J. Jwo and S.-H. Wang, “Adaptive fuzzy strong tracking extended

Kalman ﬁltering for gps navigation,” IEEE Sensors Journal, vol. 7, no. 5,

pp. 778–789, 2007.

[13] M. A. K. Jaradat and M. F. Abdel-Hafez, “Enhanced, delay dependent,

intelligent fusion for INS/GPS navigation system,” IEEE Sensors Jour-

nal, vol. 14, no. 5, pp. 1545–1554, 2014.

[14] X. Tong, Z. Li, G. Han, N. Liu, Y. Su, J. Ning, and F. Yang, “Adaptive

ekf based on hmm recognizer for attitude estimation using mems marg

sensors,” IEEE Sensors Journal, vol. 18, no. 8, pp. 3299–3310, 2017.

[15] J. He, C. Sun, B. Zhang, and P. Wang, “Adaptive error-state kalman ﬁlter

for attitude determination on a moving platform,” IEEE Transactions on

Instrumentation and Measurement, vol. 70, pp. 1–10, 2021.

[16] R. Mehra, “On the identiﬁcation of variances and adaptive Kalman

ﬁltering,” IEEE Transactions on Automatic Control, vol. 15, no. 2, pp.

175–184, 1970.

[17] L. Zhang, D. Sidoti, A. Bienkowski, K. R. Pattipati, Y. Bar-Shalom, and

D. L. Kleinman, “On the identiﬁcation of noise covariances and adaptive

Kalman ﬁltering: A new look at a 50 year-old problem,” IEEE Access,

vol. 8, pp. 59 362–59 388, 2020.

[18] H. Yan, Q. Shan, and Y. Furukawa, “RIDI: Robust IMU double integra-

tion,” in Proceedings of the European Conference on Computer Vision

(ECCV), 2018, pp. 621–636.

[19] S. Herath, H. Yan, and Y. Furukawa, “Ronin: Robust neural inertial

navigation in the wild: Benchmark, evaluations, & new methods,”

in 2020 IEEE International Conference on Robotics and Automation

(ICRA). IEEE, 2020, pp. 3146–3152.

[20] C. Chen, P. Zhao, C. X. Lu, W. Wang, A. Markham, and N. Trigoni,

“Deep-learning-based pedestrian inertial navigation: Methods, data set,

and on-device inference,” IEEE Internet of Things Journal, vol. 7, no. 5,

pp. 4431–4441, 2020.

[21] O. Asraf, F. Shama, and I. Klein, “Pdrnet: A deep-learning pedestrian

dead reckoning framework,” IEEE Sensors Journal, 2021.

PREPRINT: A HYBRID MODEL AND LEARNING-BASED ADAPTIVE NAVIGATION FILTER / OR AND KLEIN 11

[22] A. Manos, T. Hazan, and I. Klein, “Walking direction estimation

using smartphone sensors: a deep network-based framework,” IEEE

Transactions on Instrumentation and Measurement, 2022.

[23] M. Yona and I. Klein, “Compensating for partial doppler velocity log

outages by using deep-learning approaches,” in 2021 IEEE International

Symposium on Robotic and Sensors Environments (ROSE). IEEE, 2021,

pp. 1–5.

[24] D. Li, J. Xu, H. He, and M. Wu, “An underwater integrated navigation

algorithm to deal with dvl malfunctions based on deep learning,” IEEE

Access, vol. 9, pp. 82 010–82 020, 2021.

[25] M. Brossard and S. Bonnabel, “Learning wheel odometry and imu errors

for localization,” in 2019 International Conference on Robotics and

Automation (ICRA). IEEE, 2019, pp. 291–297.

[26] A. Shurin and I. Klein, “Quadnet: A hybrid framework for quadrotor

dead reckoning,” Sensors, vol. 22, no. 4, p. 1426, 2022.

[27] C. Shen, Y. Zhang, X. Guo, X. Chen, H. Cao, J. Tang, J. Li, and

J. Liu, “Seamless gps/inertial navigation system based on self-learning

square-root cubature kalman ﬁlter,” IEEE Transactions on Industrial

Electronics, vol. 68, no. 1, pp. 499–508, 2020.

[28] B. Or and I. Klein, “Learning vehicle trajectory uncertainty,” arXiv

preprint arXiv:2206.04409, 2022.

[29] J. Liu and G. Guo, “Vehicle localization during gps outages with

extended kalman ﬁlter and deep learning,” IEEE Transactions on In-

strumentation and Measurement, vol. 70, pp. 1–10, 2021.

[30] M. Brossard, S. Bonnabel, and A. Barrau, “Denoising imu gyroscopes

with deep learning for open-loop attitude estimation,” IEEE Robotics

and Automation Letters, vol. 5, no. 3, pp. 4796–4803, 2020.

[31] P. D. Groves, Principles of GNSS, Inertial, and Multisensor Integrated

Navigation Systems 2nd ed. Boston, MA: Artech House, 2013.

[32] R. G. Brown and P. Y. Hwang, Introduction to random signals and

applied Kalman ﬁltering. Wiley New York, Third Edtion, 1992.

[33] A. Shurin, A. Saraev, M. Yona, Y. Gutnik, S. Faber, A. Etzion, and

I. Klein, “The autonomous platforms inertial dataset (table 1),” IEEE

Access, vol. 10, pp. 10 191–10 201, 2022.

[34] A. L. Maas, A. Y. Hannun, A. Y. Ng et al., “Rectiﬁer nonlinearities

improve neural network acoustic models,” in Proc. icml, vol. 30, no. 1.

Citeseer, 2013, p. 3.

[35] L. Lu, Y. Shin, Y. Su, and G. E. Karniadakis, “Dying relu and initializa-

tion: Theory and numerical examples,” arXiv preprint arXiv:1903.06733,

2019.

[36] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv

preprint arXiv:1607.06450, 2016.

[37] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”

arXiv preprint arXiv:1412.6980, 2014.

[38] DJI, “Matrice 300 RTK, Aircraft Speciﬁcations,” Website source:

https://www.dji.com/matrice-300/specs, 2021.

[39] A. Shurin, A. Saraev, M. Yona, Y. Gutnik, S. Faber, A. Etzion, and

I. Klein, “The autonomous platforms inertial dataset,” IEEE Access,

vol. 10, pp. 10 191–10 201, 2022.

[40] D.-J. Jwo, J.-H. Shih, C.-S. Hsu, and K.-L. Yu, “Development of a

strapdown inertial navigation system simulation platform,” Journal of

Marine Science and Technology, vol. 22, no. 3, pp. 381–391, 2014.

Barak Or (Member, IEEE) received a B.Sc. degree

in aerospace engineering from the Technion–Israel

Institute of Technology, Haifa, Israel, a B.A. degree

(cum laude) in economics and management, and

an M.Sc. degree in aerospace engineering from the

Technion–Israel Institute of Technology in 2016 and

2018. He is currently pursuing a Ph.D. degree with

the University of Haifa, Haifa.

His research interests include navigation, deep learn-

ing, sensor fusion, and estimation theory.

Itzik Klein (Senior Member IEEE) received B.Sc.

and M.Sc. degrees in Aerospace Engineering from

the Technion-Israel Institute of Technology, Haifa,

Israel in 2004 and 2007, and a Ph.D. degree in Geo-

information Engineering from the Technion-Israel

Institute of Technology, in 2011. He is currently

an Assistant Professor, heading the Autonomous

Navigation and Sensor Fusion Lab, at the Hatter

Department of Marine Technologies, University of

Haifa. His research interests include data driven

based navigation, novel inertial navigation architec-

tures, autonomous underwater vehicles, sensor fusion, and estimation theory.