ArticlePDF Available

Efficient and Robust Freeway Traffic Speed Estimation Under Oblique Grid Using Vehicle Trajectory Data

Authors:

Abstract

Accurately estimating spatiotemporal traffic states on freeways is a significant challenge due to limited sensor deployment and potential data corruption. In this study, we propose an efficient and robust low-rank model for precise spatiotemporal traffic speed state estimation (TSE) using low-penetration vehicle trajectory data. Leveraging traffic wave priors, an oblique grid-based matrix is first designed to transform the inherent dependencies of spatiotemporal traffic states into the algebraic low-rankness of a matrix. Then, with the enhanced traffic state low-rankness in the oblique matrix, a low-rank matrix completion method is tailored to explicitly capture spatiotemporal traffic propagation characteristics and precisely reconstruct traffic states. In addition, an anomaly-tolerant module based on a sparse matrix is developed to accommodate corrupted data input and thereby improve the TSE model robustness. Notably, driven by the understanding of traffic waves, the computational complexity of the proposed efficient method is only correlated with the problem size itself, not with dataset size and hyperparameter selection prevalent in existing studies. Extensive experiments demonstrate the effectiveness, robustness, and efficiency of the proposed model. The performance of the proposed method achieves up to a 12 %\% improvement in Root Mean Squared Error (RMSE) in the TSE scenarios and an 18 %\% improvement in RMSE in the robust TSE scenarios, and it runs more than 20 times faster than the state-of-the-art (SOTA) methods.
1
Efficient and Robust Freeway Traffic Speed
Estimation under Oblique Grid using Vehicle
Trajectory Data
Yang He, Chengchuan An, Yuheng Jia, Member, IEEE, Jiachao Liu, Zhenbo Lu, and Jingxin Xia
Abstract—Accurately estimating spatiotemporal traffic states
on freeways is a significant challenge due to limited sensor
deployment and potential data corruption. In this study, we
propose an efficient and robust low-rank model for precise
spatiotemporal traffic speed state estimation (TSE) using low-
penetration vehicle trajectory data. Leveraging traffic wave priors,
an oblique grid-based matrix is first designed to transform the
inherent dependencies of spatiotemporal traffic states into the
algebraic low-rankness of a matrix. Then, with the enhanced
traffic state low-rankness in the oblique matrix, a low-rank matrix
completion method is tailored to explicitly capture spatiotemporal
traffic propagation characteristics and precisely reconstruct traffic
states. In addition, an anomaly-tolerant module based on a sparse
matrix is developed to accommodate corrupted data input and
thereby improve the TSE model robustness. Notably, driven by the
understanding of traffic waves, the computational complexity of
the proposed efficient method is only correlated with the problem
size itself, not with dataset size and hyperparameter selection
prevalent in existing studies. Extensive experiments demonstrate
the effectiveness, robustness, and efficiency of the proposed model.
The performance of the proposed method achieves up to a 12
%
improvement in Root Mean Squared Error (RMSE) in the TSE
scenarios and an 18
%
improvement in RMSE in the robust
TSE scenarios, and it runs more than 20 times faster than the
state-of-the-art (SOTA) methods.
Index Terms—Traffic state estimation, kinematic wave theory,
low-rank representation, vehicle trajectory data.
I. INTRODUCTION
A. Motivation
Precise and complete traffic states (e.g., 5-sec traffic speed)
provide reliable support for freeway proactive traffic control
and management, especially in current and future connected
and automated vehicular environments, e.g., connected and
automated vehicle (CAV) cruise control, eco-driving, and
dynamic routing planning [
1
3
]. In practice, field traffic
state measurements are often limited and noisy [
4
6
]. Fixed
This work was supported in part by the National Natural Science Foundation
of China under Grants 52272309, 52202398, and 62106044, in part by the
International Science and Technology Cooperation Project of Jiangsu Province
under Grant BZ2023015, and in part by Natural Science Foundation of Jiangsu
Province under Grant BK20210221 (Corresponding authors: Jingxin Xia).
Yang He, Chengchuan An, Zhenbo Lu, and Jingxin Xia are with the
Intelligent Transportation System Research Center, Southeast University,
Nanjing, 211189, China (e-mail: yanghe@seu.edu.cn, ccan@seu.edu.cn,
luzhenbo@seu.edu.cn, xiajingxin@seu.edu.cn).
Yuheng Jia is with the School of Computer Science and Engineering,
Southeast University, Nanjing, 211189, China (e-mail: yhjia@seu.edu.cn).
Jiachao Liu is with the Department of Civil and Environmental Engi-
neering, Carnegie Mellon University, Pittsburgh, PA 15213, USA (e-mail:
jiachaol@andrew.cmu.edu).
detectors are costly and often sparsely installed along the
road, resulting in limited spatial coverage. Mobile sensors,
benefiting from the advancements of connected vehicle (CV)
technologies, provide more extensive spatial coverage. However,
they suffer from sparsity in the temporal domain [
7
] due to
the low penetration rate in the current mixed conventional and
connected environment. Reconstructing accurate traffic states
on the freeway from the sparse and corrupted observations is
still a challenging task that needs to be addressed in current
applications of Intelligent Transportation Systems (ITSs).
B. State-of-the-Art (SOTA)
Initially, researchers carefully abstracted physical traffic
flow characteristics and utilized traffic flow models including
the first-order model like the well-known Lighthill-Whitham-
Richards (LWR) to estimate traffic states [
8
13
], employing
various data assimilation techniques. To more accurately capture
complex traffic phenomena, higher-order models such as the
Payne-Whitham (PW) models [
14
,
15
], Aw-Rascle-Zhang
(ARZ) models [
4
,
16
], and METANET models [
17
19
] have
also been explored in TSE. An alternative approach to TSE
assumes that the average speed of regular vehicles equals that
of CVs [
20
24
]. This speed-uniformity assumption simplifies
TSE by using a data-driven conservation equation model with
Kalman filters [
18
]. Recent overviews of freeway TSE highlight
these developments [
4
,
17
]. Benefiting from domain knowledge,
these methods are physically interpretable and require a
small amount of data. Despite the simplicity, model-based
methods can be constrained by the capacity of the traffic flow
models and assumptions made in the data assimilation process
[
5
]. Moreover, model-based methods usually require time-
consuming and labor-intensive parameter calibration processes.
With the rapid progress in computation ability and wide
availability of multi-source data, data-driven methods have
flourished in TSE. The main approach of this category is to
exploit the spatiotemporal dependencies from traffic data using
various learning frameworks, such as adaptive smoothing kernel
[
5
,
25
27
], Gaussian process [
28
,
29
], deep learning [
7
,
30
35
],
low-rank matrix/tensor completion [
6
,
36
,
37
], etc. The most
prevalent modeling approach is discretizing the spatiotemporal
domain into a spatiotemporal grid/matrix/diagram as shown
in Fig. 1(a). Then, fixed or mobile data are aggregated and
transformed into partial observations of the grid. The grid-based
TSE modeling has become a popular framework due to its easy
implementation and convenience in capturing high-dimensional
spatiotemporal traffic flow dependencies [6,30].
2
Fig. 1: Visualization of constructing a traffic state matrix (TSM). Traffic states exhibit high correlations along the direction
of backward traffic waves. Conventional rectangular grid-based modeling in (a) is less desirable to effectively capture such
correlations, as it simply vertically and horizontally divides the spatiotemporal region (e.g., cells A and B). In this study,
we adopted the oblique grid-based modeling in (b), strategically positioning traffic state observations along the traffic wave
direction into the same matrix column (e.g., cells C and D). This approach adeptly transforms the correlation of traffic states
into the algebraic low-rankness of the matrix, therefore ensuring a low-rank representation method to proficiently capture the
spatiotemporal correlations inherent in traffic states.
By decomposing the spatiotemporal domains into small
unified grids, Rempe et al.
[30]
developed a convolutional
neural network (CNN) to learn and reconstruct the spatiotem-
poral traffic speeds within these grids. Thodi et al.
[7]
further
incorporated kinematic wave priors into CNN by designing
anisotropic kernels to capture directional traffic propagation
characteristics. In addition, graph neural networks [
32
,
33
]
and generative adversarial networks [
34
,
35
] are also applied.
However, these deep learning-based methods may require
massive and high-quality training data. It is worth noting
that obtaining a suitable training dataset may not always be
feasible in practice [
29
]. Although the training data can be
collected from traffic simulations [
7
], the simulated dataset
may not accurately represent road segments in the real world,
depending on the quality of calibrations. To mitigate the
reliance on complete training data, physics-informed deep
learning approaches assisted by physical models have conducted
successful trials in TSE [
38
44
]. However, under conditions
of sparse data, the performance of the physics-informed deep
learning method may be sensitive to the trade-off between
model-driven and data-driven components, making reliable
training greatly challenging.
Alternatively, low-rank matrix/tensor completion, a data-
efficient grid-based data-driven approach, has emerged to deal
with limited data scenarios and achieved promising results in the
TSE domain using only sparse observations [
6
,
36
,
37
,
45
,
46
].
Based on the spatiotemporal grid/matrix, the basic idea of
this approach is to recover the spatiotemporal traffic state by
representing spatiotemporal traffic dynamic dependencies with
algebraic low-rankness. For example, Wang et al.
[6]
trans-
formed the traffic state matrix into a fourth-order Hankel tensor
and applied low-rank matrix completion on the unfolded matrix
to recover spatiotemporal traffic speeds using limited vehicle
trajectory data. Nie et al.
[37]
organized spatiotemporal traffic
speeds into a tensor and implemented spatiotemporal traffic
speeds kriging by graph-embedded tensor completion. However,
these pure data-driven low-rank representation methods may
degrade under extremely sparse data environments (e.g. 3
%
or
less vehicle trajectories).
Focusing on online applications, there are streaming-data-
driven methods that only use streaming data (e.g., real-time
data) [
4
,
47
49
]. These methods rely less on prior knowledge,
thereby demonstrating high robustness to uncertain phenomena
and unpredictable incidents. In addition to conventional fixed
and mobile sensor data, various types of interesting streaming
data are also utilized in this category, including extended
floating car data (xFCD) that can measure space and time
headway [
47
,
49
], and unmanned aerial vehicle (UAV) data
that can provide fast and accurate traffic state observations
at any desired locations in multiple travel directions [
50
53
].
However, a large amount of streaming data is usually required
for streaming-data-driven methods to provide accurate state
estimations.
C. Research Challenges and Contributions
Despite the fact that grid-based data-driven methods have
achieved high precision in previous literature, researchers
continuously contribute to this branch by tackling the following
three major challenges:
C1: consistency with backward wave propagation. Pre-
vious research has highlighted the advantages of modeling
spatiotemporal traffic characteristics along the direction of
backward waves, which propagate obliquely [
54
,
55
]. However,
most Traffic State Estimation (TSE) methods typically use an
orthogonal grid-based approach as shown in Fig. 1(a), leading
to inconsistencies with the actual propagation of non-orthogonal
backward traffic waves. As a result, these inconsistencies cause
inhomogeneous traffic states within certain grids, e.g., cells A
and B in Fig. 1(a), potentially introducing biased entries for
the TSE and diminishing its accuracy [
7
,
29
,
56
]. Furthermore,
under extremely sparse data environments, constructing the
TSM with orthogonal grids may lead to the entire column-
missing problem, which may weaken the performance of
pure data-driven models depending on column-wise algebra
similarity [
6
,
57
]. Recognizing the limitations of orthogonal
grids, He et al.
[58]
proposed oblique grids for better alignment
with traffic wave propagation, enhancing the segment-level
3
travel time estimation accuracy. For the spatiotemporal grid-
level estimation (the focus of this study), they utilized a simple
neighborhood-based imputation method, which becomes less
effective when significant data is missing. Additionally, their
approach was limited by relatively low estimation resolutions.
C2: robustness to corrupted input data. The TSE model
can be degraded when encountering unfavorable conditions
such as noisy or corrupted measurement, emphasizing the
robustness requirements against data noise and corruption. The
previous works mainly focused on the former and enhanced
their model robustness by characterizing the uncertainty caused
by stochastic disturbances in TSE [
28
,
29
]. However, random
data corruption that does not follow Gaussian distribution can
also be problematic. Though data pre-processing methods are
usually effective in removing these corrupted observations,
they might inadvertently filter out genuine observations that
are crucial for accurate traffic state estimation, depending on
hyper-parameter selection, e.g., filtering threshold. To ensure
that all potentially valuable information is utilized for accurate
state estimation, a reliable model that is robust to corrupted
raw data without destroying its integrity is desirable for TSE.
C3: computational complexity. The computational com-
plexity of exiting grid-based data-driven methods is not only
related to the problem size (i.e., temporal and spatial length
of reconstructed area) but also positively correlated with other
variables, such as the number of observations [
5
,
25
] and model
hyperparameters [
6
], bringing overwhelming computational
costs for TSE. For large-scale TSE applications with significant
problem sizes, it is practically essential to develop an efficient
model with no additional scenario-dependent or parameter-
induced computational complexity.
The existing studies have attempted to handle one or two
of the above challenges. In this study, we propose a tailored
matrix completion approach that simultaneously tackles all
these three issues. To address the C1, we integrate traffic
wave priors into a customized low-rank matrix completion
model based on the oblique grid-modeling approach by He et
al. [
58
]. The differences between their studies and our work are
as follows. First, given oblique grids, instead of exploiting the
enhanced traffic state homogeneity only, we further leverage
the enhanced algebraic low-rankness inherent in the traffic state
matrix, significantly improving TSE accuracy, especially under
severe data scarcity conditions. Second, He et al. [
58
] utilized a
simple interpolation-based imputation to estimate traffic states
with low resolutions ranging from 150m/90s to 50m/30s, while
our study proposes a tailored low-rank approach capable of
estimating high-resolution states at 3m/5s, addressing greater
challenges with an 88
%
rate of empty cells compared to 21
%
in the prior work. (2) To tackle the C2, we design an anomaly-
tolerance module to accommodate potentially corrupted traffic
state observations. Specifically, we assume the ubiquitous data
corruptions are randomly and sparsely distributed, and treat the
corrupted data detection as a sparse matrix completion problem.
(3) To respond to the C3, we employ a simple and efficient
matrix completion, in which the per-iteration computational
complexity is only related to the temporal and spatial length
of the TSE reconstructed area.
The contributions of this paper are summarized as follows:
1)
A traffic wave-inspired low-rank model is tailored for
traffic state estimation, in which an oblique grid-based
matrix is designed to enhance the low-rank nature within
the traffic states and thereby helps to proficiently capture
spatiotemporal traffic state dependencies.
2)
An anomaly-tolerant module is developed to accommodate
corrupted data input in robust traffic state estimation, with-
out requiring additional data pre-processing procedures.
3)
Theoretical computational complexity analysis and em-
pirical running time evidence prove the efficiency of
the proposed method. Numerous experiment results also
demonstrate its superior estimation accuracy and robust-
ness.
The remainder of this paper is organized as follows. Section
II gives some basic notations and defines the traffic speed
estimation problem. Section III formulates the proposed model
and derives the associated solving algorithm. Section IV
implements experiments on a real-world traffic dataset and
presents the results. Section Vpresents further discussions.
Finally, Section VI concludes this paper and provides future
research directions.
II. PRELIMINARIES
A. Notations
We use lowercase letters to denote scalars, e.g.,
aR
,
boldface lowercase letters to denote vectors, e.g.,
aRn
,
boldface capital letters to denote matrices, e.g.,
ARn1×n2
,
and Euler script letters to denote third-order tensors, e.g.,
A Rn1×n2×n3
. Given a matrix
XRn1×n2
, the matrix
nuclear norm is denoted as
X=Pmin(n1,n2)
i=1 σi(X)
,
where
σi(X)
is the
i
th largest singular value of
X
, and the
Frobenius norm is defined as
XF=qPn1
i=1 Pn2
j=1 x2
ij
.
The inner product between two matrices of the same size is
A,B= Tr ATB=qPn1
i=1 Pn2
j=1 aij bij
, where
Tr (·)
is
the matrix trace.
B. Problem description
We aim to estimate freeway traffic speeds at fixed 5-second
intervals over extended periods, using trajectory data collected
from mobile sensors such as connected vehicles (CVs). For
a single lane of the freeway segment, traffic speed variables
are collected in the spatiotemporal domain
S×W
, where
S
is
segment length and
W
is time window length. Given predefined
spatial resolution
∆s
and temporal resolution
∆t
, we can
transform the traffic state measurements into a discrete space
with matrix representation
MRL×T
, where
L=S/∆s
and
T=W/∆t
. The value of each cell is the average traffic state
variable of that cell (detailed descriptions are introduced in
subsection III-A).
The observed traffic state matrix
M
is usually incomplete
and highly sparse since the data from fixed and mobile sensors
have limited spatiotemporal coverage. In addition, the observed
entries in
M
may also contain corrupted data due to false
records and communication failures, which further complicates
the requirements of model robustness. To this end, we here
4
Fig. 2: Illustration of the proposed method. An oblique grid-based traffic state matrix is constructed (subsection
III-A
) using
incomplete and corrupted traffic state observations, and then a low-rank and sparse matrix completion model (subsection
III-B
) is applied to recover the complete low-rank spatiotemporal traffic state and to simultaneously detect potential sparse
corrupted/anomaly data.
differentiate such two levels of TSE requirements by defining
two specific tasks as follows
Traffic state estimation (TSE): to reconstruct the precise
and complete spatiotemporal traffic state from sparse but
pure observations.
Robust traffic state estimation (RTSE): to simultaneously
identify the potentially corrupted data and recover precise
and complete spatiotemporal traffic state from sparse and
corrupted (also called anomaly [59]) observations.
Note that the term ”traffic state” is used to refer to the speed
states specifically in this study.
III. METHODOLOGY
In this section, we propose an efficient and robust approach
for freeway traffic state estimation. Firstly, regarding C1,
we incorporate backward wave priors to construct an oblique
grid-based traffic state matrix in subsection
III-A
. After that,
regarding C2, we build a robust matrix completion (MC)
model to recover accurate traffic state from sparse and anomaly-
corrupted data in subsection
III-B
. Then, an Alternating Direc-
tion Method of Multipliers (ADMM)-based iterative solving
framework is elaborated in subsection
III-C
. Finally, regarding
C3, we analyze the computational complexity of the proposed
model in subsection III-D.
A. Oblique grid-based traffic state matrix construction (C1)
To construct the spatiotemporal traffic state matrix (TSM), an
intuitive idea is to virtually partition a spatiotemporal plane into
orthogonal grids (see Fig.1(a)), introducing the inconsistency
mentioned in the C1. To alleviate these inconsistencies, He et al.
[58]
proposed using non-rectangular/oblique grids to construct
spatiotemporal diagrams and proved its advantages over using
conventional rectangular grids by the improved results of
segment-level travel time estimation accuracy. However, for
the fine-grained cell-level traffic state estimation (the focus of
this study), they adopted a simple neighborhood-based iterative
imputation method to fill empty cells in the spatiotemporal
diagram, which may be sharply degraded when a large portion
of cells are missing. To address the C1 in TSE, based on
the prior work, we follow the idea of oblique grids and extend
it to fine-grained (e.g., 3m/5s) TSE under extreme missing
conditions by constructing an oblique grid-based traffic state
matrix, where the inclines of the left and right edges are aligned
with the backward wave speed, as shown in Fig. 3.
Given traffic state observations
(si, ti, xi), i = 1, .., N
,
where the
si
and
ti
are the spatial and temporal coordinates of
traffic state variable
xi
, we aim to construct a TSM along the
direction of backward traffic wave to ensure the homogeneity
within each entry of the TSM. The first step is to determine the
spatial and temporal cell index
cs
i
and
ct
i
that each observation
belongs to
cs
i=si|∆s, (1)
ct
i= (ti(bsi·tan (θ))) |∆t, (2)
where
∆s
and
∆t
are the spatial resolution and temporal
resolution used in TSM construction,
θ
is the inclined angle
of the backward wave, and
θ=arccot (v/3.6)
, where
v
is
the backward wave speed that generally ranges from -10 km/h
to -20 km/h [
58
,
60
],
b
is the intercept constant, and
b=
S·tan (θ)
, where
S
is the spatial length of the target segment.
The representative traffic state values of each cell
(l, t)
are
calculated by averaging the observed traffic state values within
the cell
¯xl,t =1
Nl,t X
cs
i=l,ct
i=t
xi,(3)
where
Nl,t
is the total number of observation points within the
cell (l, t).
B. Low-rank and Sparse Matrix Completion (C2)
Traffic states exhibit distinct spatiotemporal dependencies,
such as temporal periodicity and spatial propagation charac-
teristics shown in Fig. 1. By constructing the traffic state
matrix (TSM) with oblique grids illustrated in subsection
III-A
,
5
Fig. 3: Illustration of constructing an oblique grid-based traffic
state matrix.
the highly correlated traffic states along the backward wave
direction are strategically aligned into the same matrix column.
This alignment adeptly transforms the traffic state correlations
such as temporal recurrences and spatial dependencies into the
algebraic low-rankness of a matrix, i.e., the column-wise or
row-wise similarity. In other words, the low-rankness of the
TSM is enhanced using oblique-grid-based modeling, ensuring
a low-rank representation method to proficiently capture the
spatiotemporal correlations inherent in traffic states. This
approach enables the precise reconstruction of traffic states
from sparse observations by reformulating the TSE problem
as a low-rank matrix completion task.
Specifically, given a partially observed traffic state matrix,
the low-rank matrix completion aims to estimate the target
complete state matrix Lby minimizing its algebraic rank
min
Lrank (L)s.t. P(L) = P(M),(4)
where
M
is the partially observed traffic state matrix, and the
constraint ensures that the values of
L
and
M
are consistent
at the observation set
. Considering the rank minimization
in Eq.
(4)
is an NP-hard problem, several convex and non-
convex surrogate functions are applied to ensure computational
feasibility. In this study, we employ nonconvex truncated
nuclear norm [
61
] as the rank function, the problem in Eq.
(4)
can be rewritten as
min
LLr,s.t. P(L) = P(M),(5)
where Xr,is the truncated nuclear norm of matrix X.
However, as aforementioned in C2, the potential data cor-
ruption in traffic state observations may adversely affect the
model performance. To address the C2, we assume the data
corruptions are randomly and sparsely distributed and introduce
a sparse matrix
S
to accommodate these corruptions. A robust
Traffic Wave based Low-rank and Sparse Matrix Completion
model (TW-LSMC) is presented as
min
L,SLr,+λS1s.t. P(L+S) = P(M),(6)
where
Lr,
is the truncated nuclear norm of low-rank matrix
L
, and
S1
is the
l1
norm of sparse matrix
S
,
λ
is a weight
parameter that balances the trade-off between low-rank and
sparse regularization. In the proposed model, the traffic state
observations are represented as a combination of low-rank
structural and sparse anomaly components to simultaneously
recover the complete and accurate traffic state and detect the
anomaly data.
C. Iterative solving framework using ADMM (C2)
To reserve the original observed information in each iteration,
we do not directly update the observation matrix
M
but
introduce an auxiliary variable
W
to conduct the update and
transfer the observations from
M
to
L
and
S
. The model in
Eq.(6) is reformulated as
min
L,SLr,+λS1,
s.t. W=L+S, P(W) = P(M).(7)
To cope with the equal constraint, the augmented Lagrangian
function of our TW-LSMC model is written as
L(L,S,W,Y) = Lr,+S1+ρ
2WLS2
F(8)
+Y,WLS, s.t. P(W) = P(M),
where
⟨·,·⟩
indicates the inner product,
YRn1×n2
denotes
the Lagrangian multiplier and
ρ > 0
represents the penalty
parameter. According to the ADMM framework, the minimiza-
tion of our model can be decomposed into iteratively solving
the following three subproblems:
Ll+1 = arg min
L
LL,Sl,Wl,Yl,(9)
Sl+1 = arg min
S
LLl+1,S,Wl,Yl,(10)
Wl+1 = arg min
W
LLl+1,Sl+1 ,W,Yl,(11)
Yl+1 =Yl+ρWl+1 Ll+1 Sl+1,(12)
where
l
denotes the
l
-th iteration, and the three variables
L,S,W
are alternatively updated in each iteration until con-
vergence. The detailed solutions of Eq.
(9)
, Eq.
(10)
, and Eq.
(11)
are given in the following subsections. The pseudocode
of TW-LSMC numerical solution is summarized in Algorithm
1.
1) Update Variable
L
:Removing the irrelevant terms, the
Lsubproblem is written as
Ll+1 = arg min
L
Xr,+ρ
2
WlLSl
2
FYl,L
= arg min
L
Lr,+ρ
2
LWlSl+Yl
ρ
2
F
=DrWlSl+Yl
ρ,(13)
where
Dr
is the weighted singular value thresholding operator
as shown in Lemma 1.
Lemma 1. [
61
]For any
ρ > 0
,
ZRm×n
, and
rN+
where
r < min {m, n}
, an optimal solution to the truncated
nuclear norm minimization problem
min
XXr,+ρ
2XZ2
F,(14)
6
is given by the weighted singular value thresholding
Dr,1 (Z) = Udiag [σ
1
·1]+VT,(15)
where
Udiag (σ)VT
is the singular value decomposition of
Z
,
[·]+
denotes the positive truncation at
0
which satisfies
[σ1]+= max {σ1/ρ, 0}
,
1
{0,1}min{m,n}
is a
binary indicator vector whose first
r
entries are
0
and other
entries are 1.
2) Update Variable
S
:Specifically, the
S
subproblem is
written as
Sl+1 = arg min
S
λS1+ρ
2
WlLl+1 S
2
FYl,S
= arg min
S
λS1+ρ
2SH2
F
= sgn (H)max |H| λ
ρ,0,(16)
where
H=WlLl+1+Yl
ρ
,
indicates the point-wise product,
and the sgn (·)denotes the signum function, i.e.,
sgn (x) =
1 if x > 0,
0 if x= 0,
1 if x < 0.
(17)
3) Update Variable
W
:The
W
sub-problem is a set of
unconstrained quadratic equations element-wise. Therefore, the
closed-form solution is obtained as
Wl+1 = arg min
W
ρ
2
WLl+1 Sl+1
2
F+Yl,W
= arg min
W
ρ
2
WLl+1 +Sl+1 Yl
ρ
2
F
=Ll+1 +Sl+1 Yl
ρ,(18)
and the following transformation holds:
PWl+1=P(M),(19)
where is the observation set of spatiotemporal traffic state.
Algorithm 1: Numerical solution of Eq.
(8)
via ADMM
Input: The partially measured traffic state matrix M,
weight parameter λ, truncated parameter r.
Output: The recovered low-rank traffic state matrix L,
and sparse anomaly matrix S
Initialization: ρ= 104, ε = 104, l = 1,L=W=
M,M= mean (M),S=On1×n2, where O
denotes a matrix with all entries equal to zero ;
while not converged do
Update Ll+1 via Eq. (13) ;
Update Sl+1 via Eq. (16) ;
Update Wl+1 via Eq. (18) and (19);
Update Yl+1 via Eq. (12);
Calculate Ll+1LlF
L0
F
< ϵ;
l=l+ 1
D. Computational complexity (C3)
The computational complexity of the Algorithm 1is dom-
inated by the update of low-rank matrix
LRL×T
, which
involves a matrix truncated nuclear norm minimization problem
with respect to matrix
L
. Specifically, the
L
subproblem
only needs to solve a singular value decomposition (SVD)
of
L×T
matrix in each iteration, contributing to a per-
iteration computation complexity of
OL2T
when
L < T
.
By denoting the number of iterations by
k
, we can obtain that
the computational complexity of Algorithm 1is OkL2T.
IV. EXP ER IM EN TS
In this section, we evaluate our proposed TW-LSMC method
on real-world traffic dataset in comparison with state-of-the-
art methods, which are summarized to answer the following
research questions (RQs):
RQ1 (
IV-C
): How about the performance of the proposed
TW-LSMC in sparse data environments?
RQ2 (
IV-D
): How about the performance of the proposed
TW-LSMC with corrupted data input?
RQ3 (
IV-E
): How does the wave speed parameter of TW-
LSMC affect the TSE performance?
RQ4 (
IV-F
): How do different model components con-
tribute to model performance?
RQ5 (
IV-G
): How about the computational efficiency of
the proposed model compared to existing SOTA methods?
A. Data description and corrupted data generation
In this study, we use vehicle trajectories extracted from video
cameras on lane 2 of US Highway 101 of the NGSIM dataset.
Similar to the previous work by Wang et al.
[6]
, our experiments
cover a segment of 621 meters, and the test duration is 2400
seconds. We focus on the traffic state with a resolution of
3 meters and 5 seconds, where the traffic state is defined as
the average vehicle speed in each grid cell. Consequently, the
spatiotemporal size of the traffic state matrix is
207 ×480
. The
traffic speed maps of the entire dataset are shown in Fig. 4(a).
To evaluate the model performance on robust traffic state es-
timation, we design two types of non-Gaussian data corruption
that may adversely affect the TSE performance:
Type I: the observed data under the free-flow state are
tampered to the jam waves/stop-and-go waves state.
Type II: the observed data under the jam waves/stop-and-
go waves state are tampered to the free-flow state.
These two types of corruption introduce false information and
can greatly affect the estimation of the surrounding traffic state.
We define the tampered speed of two types of corruption as
follows
vI=vf50,(20)
vII =vc+ 80,(21)
where the
vf50
km/h and
vc5
km/h are the actual speed
observations under free-flow and jam waves/stop-and-go waves
state [62].
7
B. Baseline models and evaluation metrics
We compared the proposed TW-LSMC model with the
following six alternative methods:
LSMC (Low-rank and Sparse Matrix Completion, [
63
]):
A rectangular grid-based low-rank and sparse matrix com-
pletion method with truncated nuclear norm minimization
[61] and l1norm minimization.
LWR-CG (LWR model-based Computational Graph,
[
44
]): A multi-source data compatible computational
graph approach incorporating the LWR model [
64
,
65
],
three-detector model [
54
], and fluid queue model for
traffic state and queue profile joint estimation. As only
vehicle trajectory data is used in this study, the first two
physical models are mainly operational.
ASM (Adaptive Smoothing Method, [
25
]): a spatiotempo-
ral kernel-weighted method that considers free-flow and
congested traffic wave propagation characteristics.
SD-EGTF/SD-ASM (Shear/Oblique Grid-based Discrete
Extended Generalised Treiber–Helbing Filter (EGTF),
[
56
]): An oblique grid-based EGTF [
66
] speed state
estimator for virtual vehicle trajectory generation. As
only one data source (e.g., vehicle trajectories) is used
in this study, the EGTF degrades to the Generalised
Treiber–Helbing Filter (i.e., Adaptive Smoothing Method)
[
25
,
67
]. For clarity, we denote the SD-EGTF as SD-ASM
in the following sections.
PSM (Phase-based Smoothing Method, [
27
]): A kernel-
weighted smoothing method based on Kerner’s three-phase
theory [68].
STH-LRTC (Spatiotemporal Hankel Low-Rank Tensor
Completion, [
6
]): A low-rank tensor completion with the
spatiotemporal Hankelization to reconstruct the spatiotem-
poral traffic speed.
The hyperparameters in each model greatly affect the TSE
performance. For a fair comparison, the baseline models
are fine-tuned. For the ASM model, we set the parameters
according to the suggested values in [
6
,
25
]. Specifically, the
wave speeds are set as
vf
= 60 km/h and
vc
= -10 km/h, the
kernel parameters are
σ= 200m, τ = 10s
, and the weighted
parameters are
V= 10
km/h and
Vthr = 20
km/h. For
STH-LRTC, the parameter setting
τs= 40, τt= 30
is used to
obtain the Hankel tensor in Wang et al.
[6]
. However, we find
that this setting provides poor estimation results in some cases.
According to the parameter grid search results, we set the
embedding length that achieved the best overall performance
for each scenario in our experiments, as noted in Tab. I. For the
PSM, we set the speed thresholds
Vthr
J= 25
km/h,
Vthr
S= 65
km/h,
Vthr
F= 55
km/h, smoothing directions
Vdir
J,S =18
km/h,
Vdir
F= 70
km/h, kernel parameters
τS,F = 20
s,
τH
F,S = 20
s,
σF,S = 100
m as suggested in Rempe et al.
[27]
. For the LWR-
CG, we set the weight of partial differential equations (PDE) as
1, the learning parameter as
104
, and the number of epochs
as 10000. The distributed computing framework is utilized
in the separated periods [0s,1200s] and [1200s, 2400s] due
to computing memory constraints. For the proposed method,
we use truncated percentage parameter
θ= 0.3
, weighted
parameter
λ= 0.04
, and learning rate control parameter
ρ= 104, as illustrated in Algorithm 1.
To guarantee fair comparisons, all experiments are conducted
on a desktop with a 3.7 GHz Intel Core i5-9600 K processor
and 32 GB of RAM. The STH-LRTC, ASM, and SD-ASM
are implemented using Matlab R2018b. The LWR-CG is
implemented using Python with TensorFlow-2.10.0. The PSM
is coded using Python 3.8 with Numpy-1.19.2 and Pytorch-
1.9.0. The LSMC and proposed TW-LSMC are coded in
Python 3.8 using NumPy-1.19.2 only. The code is available at
https://github.com/heyang49/TW-LSMC.
The partially observed speed data from trajectories are used
to recover the full traffic speed in the following TSE and RTSE
experiments. Specifically, we randomly select trajectories as
input data. We use Root Mean Squared Error (RMSE) and
Mean Absolute Error (MAE) as evaluation metrics to evaluate
the performance of different models under TSE and RTSE
scenarios.
RMSE = r1
nXn
i=1 (yiˆyi)2,(22)
MAE = 1
nXn
i=1 |yiˆyi|,(23)
where
n
is the number of test data,
yi
is the ground truth
and
ˆyi
is the estimation. Note that the ground truth speed is
calculated from all the trajectory points within the grid cell.
C. Traffic State Estimation (RQ1)
To assess the TSE model performance, we begin by visu-
alizing the estimation results of the proposed and alternative
methods under a 5 % CV penetration rate scenario using the
NGSIM data. Fig. 4(a) displays the ground truth traffic speed
matrix, depicting intricate traffic dynamics evolution with mul-
tiple shockwaves, thereby making it desirable for performance
evaluation. Fig. 4(b) shows a training dataset chosen from 20
independent experiments, highlighting significant data missing
during certain intervals, such as between 950s to 1225s and
1750s to 2000s, which complicates the task for models to
accurately reconstruct traffic speeds.
Fig. 4(c) visualizes the results using the vanilla LSMC,
where LSMC’s state estimates are significantly deficient in
the columns that speed observations are entirely missing,
primarily because the standard low-rank technique relies
heavily on column/row-wise similarities, i.e., algebraic low-
rankness. Leveraging partial differentiation equations, the
physics-informed LWR-CG method (seen in Fig. 4(d)) offers
continuous state estimations and depicts congestion patterns, but
struggles to precisely reconstitute shockwaves in predominantly
missing areas. By applying isotropic smoothing kernels based
on the two-phase [
54
] and three-phase [
68
] wave theory, the
ASM (Fig. 4(e)) and the PSM (Fig. 4(f)) reconstruct clearer
shockwaves than the LWR-CG. The speed estimations of
ASM in the jam area tend to be lower than actual due to
the smoothing effects, a limitation mitigated by PSM which
offers refined speed estimates. Comparatively, PSM notably
outperforms ASM, particularly in the jam and transition areas,
owing to its integration of a synchronized flow phase. The
STH-LRTC (Fig. 4(g)) surpasses both ASM and PSM in
accuracy. However, during the period with limited observations
8
Fig. 4: A TSE experiment on the NGSIM dataset: (a) The ground truth traffic speed; (b) The observed traffic speed from 5
%
randomly
selected vehicle trajectories; (c) The estimation result by LSMC; (d) The estimation result by LWR-CG; (e) The estimation result by ASM;
(f) The estimation result by PSM; (g) The estimation results by STH-LRTC; (h) The estimation results by the proposed TW-LSMC.
(see blue rectangles), both STH-LRTC and smoothing models
inadequately estimate shockwaves. In contrast, the proposed
TW-LSMC approach showcased in Fig. 4(h) successfully
reconstructs both major and minor shockwaves with fine-
grained features, such as accurate wave lengths, and clear wave
boundaries, demonstrating remarkable robustness to sparse data.
Driven by an understanding of traffic wave behaviors, the TW-
LSMC identifies highly correlated traffic states generated by
the same backward wave and builds connections among these
states, particularly in distant positions, through a low-rank
framework.
By comparing Fig. 4(c), (g) with Fig. 4(h), it is evident that
the proposed oblique grid-based TW-LSMC adeptly captures
distinct traffic propagation characteristics such as stop-and-
go shockwaves, which conventional low-rank-based LSMC
and STH-LRTC methods cannot model. This leads to remark-
able enhancements in estimation accuracy, exemplified by a
reduction of 6.73 in RMSE when compared to the rectangular
grid-based LSMC. These results confirm the necessity of incor-
porating traffic wave priors and the effectiveness of the oblique
grid in enhancing traffic state low-rankness. Furthermore, the
improved low-rankness rendered by the TW-LSMC not only
enhances its accuracy over the purely data-driven STH-LRTC
approach but also improves its efficiency. Detailed discussions
on the theoretical complexity analysis and supporting empirical
evidence are provided in subsection IV-G. By comparing Fig.
4(e), (f) with Fig. 4(h), we can observe that incorporating traffic
wave priors into two distinct modeling approaches, the low-
rank-based TW-LSMC outperforms smoothing-based the ASM
and PSM approaches, indicating the superiority of low-rank
representation in learning inherent traffic state dependencies.
The SD-ASM exhibits overall similar estimation effects to
ASM, with their primary differences shown in local perspectives
due to their utilization of different grid structures. Consequently,
SD-ASM is not depicted in Fig. 4. Instead, Fig. 5zooms in
on the nuanced differences between ASM and SD-ASM to
more clearly illustrate the impact of employing oblique versus
rectangular grids. The visualized period in Fig. 5is from
the 950s to 1225s, corresponding to the left blue rectangle
in Fig. 4. As depicted in Fig. 5(a), the ASM, which uses a
rectangular grid, is prone to a noticeable aliasing effect, leading
to speed discontinuities. In contrast, the SD-ASM, which adopts
an oblique grid under the same spatiotemporal resolution,
presents a significantly smoother profile, shown in Fig. 5(b).
This enhancement in performance is further supported by
reductions in the average RMSE/MAE and standard deviation
as detailed in Tab. I, highlighting the effectiveness of the
oblique grid in delivering consistent estimates and promoting
state homogeneity. Furthermore, A direct comparison between
Fig. 5(b) and Fig. 5(c) reveals that within the same oblique
9
Fig. 5: Comparison between rectangular and oblique grid-based
methods (Zoom In).
grid framework, the proposed low-rank-based TW-LSMC
reconstructs more complete and precise shockwaves than the
smoothing-based SD-ASM, showcasing the superior capability
of TW-LSMC.
To comprehensively evaluate the model performance, we
design multiple testing scenarios with varying levels of vehicle
penetration rates. Specifically, we configure the penetration
rates of connected vehicles (CVs) as 3
%
, 5
%
, 10
%
, and 15
%
,
and repeat the experiment 20 times in each CV penetration
scenario by randomly selecting different vehicle trajectories.
Tab. Isummarizes the RMSE (km/h) and MAE (km/h) with
standard deviations of all models, numerically demonstrating
their TSE performance. Overall, the proposed TW-LSMC
outperforms the baseline models, particularly showing strength
in scenarios with lower CV penetration rates, such as 3
%
and
5
%
. In comparison to the ASM, the SD-ASM shows enhanced
performance in terms of both accuracy and reduced variability
across all scenarios, attributed to the integration of the oblique
grid, which effectively addresses speed inconsistencies. Mean-
while, the PSM, augmented with a synchronized phase-based
kernel, surpasses the ASM in penetration scenarios from 5
%
to 15
%
. However, its performance dips below that of the ASM
at the lowest penetration rate of 3
%
, probably because more
complex models usually require more data for training.
Notably, under the extremely sparse data environment of
3
%
CV penetration, the pure data-driven method STH-LRTC
degrades sharply. This is because the Hankelization operation
in STH-LRTC, which only integrates the limited observations
from a surrounding orthogonal area of size
τs×τt
, struggles
to capture sufficient data under such extreme conditions. Con-
versely, the proposed TW-LSMC continues to provide accurate
speed estimation results, offering more reliable support for
refined proactive traffic control and management applications.
Therefore, in the early stage of mixed conventional and
connected environments with low CV penetration, the proposed
TW-LSMC emerges as the more appropriate option.
D. Robust Traffic State Estimation (RQ2)
To directly showcase the robustness of TSE models, we
initiate our evaluation with a certain data corruption scenario.
Fig. 6: An RTSE experiment on the NGSIM dataset: (a) The observed
traffic speed; (b) The estimation result by ASM; (c) The estimation
results by STH-LRTC; (d) The recovered low-rank traffic state matrix
by the proposed TW-LSMC; (e) The recovered sparse anomaly matrix
by the proposed TW-LSMC, positive entries (blue): type II data
corruptions, negative entries (red): type I data corruptions.
Fig. 6presents the estimation results of the proposed TW-
LSMC alongside two state-of-the-art (SOTA) baseline methods.
Given that the ASM, SD-ASM, and PSM methodologies
are all based on smoothing techniques and that the PDEs
utilized in the LWR-CG model have similar effects to the
smoothing kernel, we choose ASM as the representative
method to depict the robust TSE performance for this group.
Fig. 6(a) shows the observed traffic speed of 10
%
randomly
selected trajectories with 30 type I and II data corruptions
defined in Eq.
(20)
and Eq.
(21)
. The ASM’s estimations,
depicted in Fig. 6(b), show a relative insensitivity to corruption,
attributed to the anomaly-mitigating effect of its weighted
smoothing operation. When compared to ASM, the STH-LRTC
10
TABLE I: TSE performance comparison in average RMSE (km/h) and MAE (km/h) with the standard deviation.
RMSE (km/h) MAE (km/h)
CV-3% CV-5% CV-10% CV-15% CV-3% CV-5% CV-10% CV-15%
LSMC [63] 15.18 ± 0.94 13.74 ± 0.72 11.29 ± 0.67 9.42 ± 0.60 13.15 ± 0.71 10.65 ± 0.61 8.40 ± 0.49 6.90 ± 0.40
LWR-CG [44] 10.75 ± 0.56 8.52 ± 0.42 6.91 ± 0.35 6.55 ± 0.18 7.54 ± 0.31 6.32 ± 0.27 5.45 ± 0.22 4.91 ± 0.16
ASM [25] 9.89 ± 0.65 8.27 ± 0.51 6.86 ± 0.32 6.45 ± 0.19 7.27 ± 0.40 6.09 ± 0.36 5.12 ± 0.20 4.85 ± 0.14
SD-ASM [56] 9.82 ± 0.29 8.20 ± 0.20 6.77 ± 0.10 6.36 ± 0.03 7.25 ± 0.15 6.08 ± 0.11 5.07 ± 0.04 4.78 ± 0.02
PSM [27] 10.85 ± 3.58 8.08 ± 0.99 6.63 ± 0.33 6.34 ± 0.32 7.44 ± 1.94 5.93 ± 0.56 4.99 ± 0.25 4.73 ± 0.23
STH-LRTC [6] 36.06 ± 3.65 8.66 ± 2.84 5.89 ± 1.26 5.04 ± 0.32 23.6 ± 2.10 6.10 ± 1.37 4.37 ± 1.04 3.72 ± 0.18
TW-LSMC 9.53 ± 0.75 7.56 ± 0.55 5.76 ± 0.44 5.14 ± 0.25 7.13 ± 0.45 5.66 ± 0.46 4.30 ± 0.24 3.86 ± 0.16
Delay-embedding lengths: aτs= 60,τt= 60,bτs= 40,τt= 50,cτs= 30,τt= 50,dτs= 20,τt= 50.
method yields more accurate results in areas unaffected by
corruption. However, its performance significantly declines
within corrupted zones (see the blue rectangles in Fig. 6(c)),
owing to the presumption of uncorrupted speed observations
in the Hankel tensor construction. Fig. 6(d) and (e) show
the TW-LSMC’s reconstructed low-rank traffic state matrix
and the sparse anomaly matrix respectively. The positive
and negative values in Fig. 6(e) refer to the type II and I
data corruptions, respectively. The low-rank matrix accurately
provides complete structural traffic states, while the sparse
matrix successfully detects both types of randomly injected
corruptions (see the blue rectangles Fig. 6(e)), confirming the
necessity of individually modeling the potential anomalies in
a robust TSE model.
To comprehensively evaluate the model performance of
robust traffic state estimation (RTSE) under varying data
corruption levels, we randomly inject a variety number of
type I and type II data corruptions into observations. Fig. 7
displays the performance (in RMSE) of the proposed and
alternative methods. As the corruption level increases, our
TW-LSMC model which leverages a low-rank and sparse
representation exhibits remarkable robustness with RMSE
values rising modestly from 5.5 to 6.5 km/h. In contrast, the
performance of the low-rank Hankel tensor-based STH-LRTC
method deteriorated significantly, with RMSE increasing from
6.0 to 8.0 km/h. These results demonstrate the effectiveness
of the anomaly-tolerant module in the proposed method. In
the meantime, ASM performs insensitivity to the changes in
corruption level, with RMSE increasing from 7.0 to 8.0 km/h,
because the smoothing operation in ASM can mitigate the
negative effect of anomalies to a certain extent. The inadequate
performance of ASM mainly stems from the basic estimation
ability in anomaly-free scenarios, which is due to the limitation
of smoothing operation’s capability to capture traffic state
dependencies, as previously discussed in subsection IV-C.
E. Sensitivity analysis (RQ3)
The backward wave speed is an important parameter in the
proposed TW-LSMC, as it introduces valuable physical traffic
propagation knowledge. The significance of traffic wave prior is
tested in the ablation study (subsection
IV-F
). We investigate the
wave speed sensitivity of the proposed method under different
CV penetrations using the NGSIM dataset and summarize the
model performance (in RMSE) in Fig. 8. The backward wave
speeds around the world generally range from -10 to -20 km/h
Fig. 7: Robust TSE performance comparison under varying data
corruption levels. For each corruption level, the number of type I and
type II data anomalies/corruptions is the same.
Fig. 8: Sensitivity of the wave speed on NGSIM dataset.
[
58
,
60
]. For three CV penetration scenarios, the TW-LSMC
achieved the best performance with the lowest RMSE when the
wave speed equals -18 km/h, indicating the actual backward
wave speed of the NGSIM dataset, which is consistent with the
estimated value in [
69
,
70
]. It is also interesting to find that the
performance of the TW-LSMC reaches a stable platform when
the absolute value of the wave speed parameter is larger than
16 km/h, suggesting a recommended wave speed value range
that provides acceptable performance. It is another indication
of the robustness of the proposed method, as a stable high-
performance interval of the core model parameter is useful in
practical scenarios.
11
TABLE II: Average RMSE (km/h) and MAE (km/h) with
standard deviation of variant methods in ablation study.
Metric
Method
TW-LSMC w/o TW (LSMC) w/o nonconvex w/o S term
RMSE 6.07 ±0.43 11.40 ±0.51 7.13 ±0.86 6.64 ±0.41
MAE 4.47 ±0.24 8.50 ±0.38 5.13 ±0.45 4.80 ±0.25
F. Ablation study (RQ4)
To inspect the significance of each component of the
proposed method, we conduct an ablation study to compare the
performance of model variation by repeating the experiments
20 times on the RTSE scenarios using 10
%
trajectories with 30
type I and type II data corruptions. We examine three variations
of the proposed method: (1) In TW-LSMC w/o TW, we adopt
the conventional matrix construction instead of the one using
the traffic wave prior. (2) In TW-LSMC w/o nonconvex, we
replace the nonconvex truncated nuclear norm (TNN) function
with the convex nuclear norm (NN) function defined in the
subsection
II-A
. (3) In TW-LSMC w/o S term, we remove
the sparse anomaly components in TW-LSMC, and only the
low-rank matrix is preserved. Results of different variations
are shown in Tab. II.
From the results, we can observe that the performance of
all the variations degraded, indicating that each component
contributes to the overall improvement of TW-LSMC remark-
ably. After replacing the nonconvex TNN term, the errors show
an obvious increase, demonstrating that the nonconvex rank
surrogate function is more capable of capturing the traffic
state’s low-rank nature (i.e., spatiotemporal dependencies) than
the convex one. It is notable that without traffic wave prior, the
accuracy decreases sharply. This indicates that the vanilla low-
rank matrix completion method is incapable of capturing the
traffic dynamics and propagation characteristics. The ablation
studies on sparse matrix
S
term manifest that potentially
corrupted data should be considered and modeled in the TSE
model, and the increments of RMSE and MAE verify this
finding.
G. Computation Performance (RQ5)
To demonstrate the computational performance of the pro-
posed TW-LSMC model, we first theoretically analyze the
temporal computational complexity of the proposed and four
representative baseline methods in Tab. III, and then give
empirical running time evidence in Tab. IV.
1) Computational complexity comparison: Before analyzing
the computational complexity, we denote the spatial and
temporal length of the input traffic state matrix by
L
and
T
and
the number of iterations by
k
. The most time-consuming step
when training LWR-CG is within the shared layer that connects
the input layer with subsequent layers, which contributes to a
complexity
O(kbLT M )
, where
M
is the number of hidden
neurons,
b
is the batch size, and
k
is the number of epochs.
The complexity of ASM is dominated by the calculation of
two free-flow and congested speed fields. For each single
spatiotemporal location
(l, t)
, all the data observations are
used for the calculation. Thus, the complexity of ASM is
TABLE III: The computational complexity of the baseline and
proposed models.
Method Computational complexity
LWR-CG O(kbLT M)a
ASM/SD-ASM O(N LT )b
PSM O(LT τ σ)c
STH-LRTC O2
sτ2
t(Lτs+ 1) (Tτt+ 1)d
TW-LSMC OkL2Te
ak= 10000,L= 207,T= 480,b=N/3, M = 125.
bL= 207
,
T= 480
(ASM),
T= 505
(SD-ASM),
N=
2980 14904.
cL= 207,T= 480,τ= 20 500, σ = 100 1000.
dk= 80 100,L= 207,T= 480,τs= 20 60,τt= 50.
ek= 50 100,L= 207,T= 505.
O(N LT )
, where
N
is the number of observations. As a result,
the computation time of ASM will dramatically increase when
more high-resolution data are used. The SD-ASM shares the
same complexity with ASM. The most complex step in PSM
is the convolution process when calculating phase-dependent
speeds, contributing to a complexity
O(LT τ σ)
, where
τ
and
σ
are the temporal and spatial lengths of the convolution
kernel. For the STH-LRTC, the most complex step is the
update of the Hankel tensor
X Rτs×τt×(Lτs+1)×(Tτt+1)
,
where
τs
and
τt
are the embedding lengths of Hankel tensor.
Specifically, it applies one SVD on the reshaped matrix
X
Rτsτt×(Lτs+1)(Tτt+1)
, contributing to a per-iteration com-
putational complexity of
Oτ2
sτ2
t(Lτs+ 1) (Tτt+ 1)
.
Therefore, the computational complexity of STH-LRTC will
increase when larger embedding lengths are configured in the
Hankel tensor. The computational complexity of the proposed
method is analyzed in subsection III-D.
In our experiments, the spatiotemporal size of the input
traffic state matrix
L
and
T
, the number of required convergence
iterations
k
, the number of data observations
N
, the embedding
lengths
τt
and
τs
, and the kernel lengths
τ
and
σ
are noted at
the bottom of Tab. III. To ensure an identical spatiotemporal
reconstruction area, the temporal size of the oblique grid-based
input TSM is slightly larger than the orthogonal grid-based
TSM, i.e.,
505
and
480
. As the spatial length
L
is much smaller
than the number of observations
N
in ASM and the squared
embedding length
τ2
sτ2
t
in STH-LRTC, we theoretically prove
that our method is more computationally efficient than the
state-of-the-art (SOTA) data-driven methods.
2) Running time comparison: To further compare the com-
putational efficiency of TSE models, we summarize the running
time of the proposed and baseline models under various CV
penetrations in Tab. IV. Overall, the total running time of
the proposed TW-LSMC consistently outperforms the SOTA
models in all scenarios, demonstrating its promising and reliable
computational ability regardless of missing data characteristics.
Among all the models evaluated, the LWR-CG requires the
longest training time, which can be attributed to its extensive
number of iterations and batch size. The running time of ASM
dramatically grows with the increase of the CV penetration
since the computational complexity of ASM is positively related
to the amount of data. In contrast, the running time of STH-
LRTC decreases, because the Hankel tensor can be smaller
12
TABLE IV: The average running time (s) with the standard
deviation in NSGIM data experiment under various CV
penetrations.
Scenarios
Method
LWR-CG ASM PSM STH-LRTC TW-LSMC
CV-3% 6430.2 ± 70.3 24.8 ± 2.3 24.8 ± 1.3 1997.3 ± 162.3 0.85 ± 0.06
CV-5% 8742.6 ± 83.7 42.9 ± 3.4 28.1 ± 1.5 524 ± 19.0 0.81 ± 0.02
CV-10% 11384.1 ± 96.1 84.6 ± 4.4 42.1 ± 2.0 337.8 ± 3.1 0.78 ± 0.02
CV-15% 13215.8 ± 103.0 122.7 ± 4.5 55.7 ± 2.7 185.7 ± 1.6 0.77 ± 0.06
when input data are more sufficient, e.g,
τs= 60
m is used in
the CV-3
%
case and
τs= 20
m is applied in the CV-15
%
case.
The running time of PSM is smaller than ASM because only
the speed observations within convolution kernels are used
in PSM when estimating phase-dependent speeds, instead of
using all observations in ASM. In summary, the theoretical
analysis and empirical evidence both confirm the computational
superiority of the proposed method.
V. DISCUSSION
A. Use conditions
The present work proposes a traffic wave-based low-rank
and sparse matrix completion model that utilizes trajectory data
obtained from connected vehicles (CVs). The proposed model
is also compatible with fixed detector data, as any observations
can be transformed into the input entries of the traffic state
matrix to improve the state estimation accuracy. The most
important aspects of this study are: (1) No physical traffic
model is used, only a backward traffic wave speed is required,
which generally ranges from -10 km/h to -20 km/h around the
world [
58
,
60
] and the model performance is robust to this
parameter selection when the absolute wave speed parameter
larger than 16km/h; (2) No data pre-processing procedures are
required, and the model can accommodate corrupted input data;
(3) No extensive historical data are required for model training,
i.e., the model is unsupervised; (4) Only small penetration
rates, e.g., 5
%
CV deployed in the freeway, are sufficient to
provide traffic speed estimations with small errors.
B. Limitations
As discussed, the applicability of the current research is
straightforward. One limitation of this study is only traffic
speed states are being estimated. The direct application of
the proposed methodology employing CV trajectories for
traffic density estimation may be biased. This is due to the
deviated traffic density observations measured from the CVs,
compounded by the absence of additional physical models (e.g.,
fundamental diagram). A viable strategy to resolve it involves
the integration of connected and automated vehicles (CAVs) that
allow the collection of space or time headway from surrounding
vehicles, thereby enabling the generation of unbiased traffic
density measurements within small spatiotemporal grid cells,
exemplified by cells of 3 meters by 5 seconds.
VI. CONCLUSION
In this study, we propose a simple and efficient matrix
completion model for traffic state estimation (TSE) using sparse
vehicle trajectory data. Inspired by the traffic wave prior, we
construct the traffic state matrix with oblique grids to capture
the recurrent traffic dynamics and directional traffic propagation
characteristics. To enhance the robustness of the proposed
TSE model, we design an anomaly-tolerant module to detect
and remove anomalies in traffic state observations. Extensive
experiments indicate that (1) the oblique grid-based modeling is
able to capture traffic dynamics and achieves reliable estimation
performance, especially in extremely sparse data conditions,
(2) the model consistently performs robustness to various data
corruption levels, and (3) the model is robust to wave speed
parameters, can adapt to diverse traffic scenarios and is more
computationally efficient than the SOTA data-driven methods.
There are several further directions for future study. First,
the present model is designed for the traffic speed estimation
problem and could be extended to estimate volume, density,
and other traffic state variables. Second, while this study
addresses random non-Gaussian data corruption (C2), future
investigations could explore more intentional corruption such
as cyber-attacks. Third, the proposed method is evaluated using
a connected vehicle (CV) trajectory dataset. Future endeavors
could look into applying this methodology to multi-source
traffic datasets or extended floating car data (xFCD) from
connected and automated vehicles (CAVs).
REFERENCES
[1]
K. Boriboonsomsin, M. J. Barth, W. Zhu, and A. Vu, “Eco-routing
navigation system based on multisource historical and real-time traffic
information,” IEEE Transactions on Intelligent Transportation Systems,
vol. 13, no. 4, pp. 1694–1704, 2012.
[2]
E. Ozatay, S. Onori, J. Wollaeger, U. Ozguner, G. Rizzoni, D. Filev,
J. Michelini, and S. Di Cairano, “Cloud-based velocity profile opti-
mization for everyday driving: A dynamic-programming-based solution,
IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 6,
pp. 2491–2505, 2014.
[3]
J. Guanetti, Y. Kim, and F. Borrelli, “Control of connected and automated
vehicles: State of the art and future challenges,” Annual reviews in control,
vol. 45, pp. 18–40, 2018.
[4]
T. Seo, A. M. Bayen, T. Kusakabe, and Y. Asakura, “Traffic state
estimation on highway: A comprehensive survey,” Annual reviews in
control, vol. 43, pp. 128–151, 2017.
[5]
C. Yang, B. T. Thodi, and S. E. Jabari, “Generalized adaptive smoothing
using matrix completion for traffic state estimation, in 2022 IEEE 25th
International Conference on Intelligent Transportation Systems (ITSC).
IEEE, 2022, pp. 787–792.
[6]
X. Wang, Y. Wu, D. Zhuang, and L. Sun, “Low-rank hankel tensor
completion for traffic speed estimation, IEEE Transactions on Intelligent
Transportation Systems, vol. 24, no. 5, pp. 4862–4871, 2023.
[7]
B. T. Thodi, Z. S. Khan, S. E. Jabari, and M. Men
´
endez, “Incorporating
kinematic wave theory into a deep learning method for high-resolution
traffic speed estimation, IEEE Transactions on Intelligent Transportation
Systems, 2022.
[8]
Y. Yuan, J. Van Lint, R. E. Wilson, F. van Wageningen-Kessels, and S. P.
Hoogendoorn, “Real-time lagrangian traffic state estimator for freeways,
IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 1,
pp. 59–70, 2012.
[9]
Y. Yuan, H. Van Lint, F. Van Wageningen-Kessels, and S. Hoogendoorn,
“Network-Wide Traffic State Estimation Using Loop Detector and Floating
Car Data,” Journal of Intelligent Transportation Systems, vol. 18, no. 1,
pp. 41–50, Jan. 2014.
[10]
D. B. Work, S. Blandin, O.-P. Tossavainen, B. Piccoli, and A. M. Bayen,
“A Traffic Model for Velocity Data Assimilation,” Applied Mathematics
Research eXpress, vol. 2010, no. 1, pp. 1–35, Jan. 2010.
[11]
R. Wang, S. Fan, and D. B. Work, “Efficient multiple model particle
filtering for joint traffic state estimation and incident detection,”
Transportation Research Part C: Emerging Technologies, vol. 71, pp.
521–537, Oct. 2016.
13
[12]
R. Wang, D. B. Work, and R. Sowers, “Multiple Model Particle Filter
for Traffic Estimation and Incident Detection, IEEE Transactions on
Intelligent Transportation Systems, vol. 17, no. 12, pp. 3461–3470, Dec.
2016.
[13]
A. Duret and Y. Yuan, “Traffic state estimation based on eulerian
and lagrangian observations in a mesoscopic modeling framework,
Transportation research part B: methodological, vol. 101, pp. 51–71,
2017.
[14]
C. Nanthawichit, T. Nakatsuji, and H. Suzuki, Application of Probe-
Vehicle Data for Real-Time Traffic-State Estimation and Short-Term
Travel-Time Prediction on a Freeway,” Transportation Research Record:
Journal of the Transportation Research Board, vol. 1855, no. 1, pp.
49–59, Jan. 2003.
[15]
Y. Liu, S. He, B. Ran, and Y. Cheng, “A Progressive Extended
Kalman Filter Method for Freeway Traffic State Estimation Integrating
Multisource Data,” Wireless Communications and Mobile Computing,
vol. 2018, pp. 1–10, 2018.
[16]
R. Wang, Y. Li, and D. B. Work, “Comparing traffic state estimators
for mixed human and automated traffic flows, Transportation Research
Part C: Emerging Technologies, vol. 78, pp. 95–110, May 2017.
[17]
Y. Wang, M. Zhao, X. Yu, Y. Hu, P. Zheng, W. Hua, L. Zhang, S. Hu,
and J. Guo, “Real-time joint traffic state and model parameter estimation
on freeways with fixed sensors and connected vehicles: State-of-the-art
overview, methods, and case studies,” Transportation Research Part C:
Emerging Technologies, vol. 134, p. 103444, Jan. 2022.
[18]
M. Zhao, C. Roncoli, Y. Wang, N. Bekiaris-Liberis, J. Guo, and
S. Cheng, “Generic Approaches to Estimating Freeway Traffic State
and Percentage of Connected Vehicles With Fixed and Mobile Sensing,
IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 8,
pp. 13 155–13 177, Aug. 2022.
[19]
M. A. Makridis and A. Kouvelas, An adaptive framework for real-
time freeway traffic estimation in the presence of CAVs,” Transportation
Research Part C: Emerging Technologies, vol. 149, p. 104066, Apr. 2023.
[20]
N. Bekiaris-Liberis, C. Roncoli, and M. Papageorgiou, “Highway Traffic
State Estimation With Mixed Connected and Conventional Vehicles,
IEEE Transactions on Intelligent Transportation Systems, vol. 17, no. 12,
pp. 3484–3497, Dec. 2016.
[21]
C. Roncoli, N. Bekiaris-Liberis, and M. Papageorgiou, “Use of Speed
Measurements for Highway Traffic State Estimation: Case Studies on
NGSIM Data and Highway A20, Netherlands,” Transportation Research
Record: Journal of the Transportation Research Board, vol. 2559, no. 1,
pp. 90–100, Jan. 2016.
[22]
M. Fountoulakis, N. Bekiaris-Liberis, C. Roncoli, I. Papamichail,
and M. Papageorgiou, “Highway traffic state estimation with mixed
connected and conventional vehicles: Microscopic simulation-based
testing,” Transportation Research Part C: Emerging Technologies, vol. 78,
pp. 13–33, May 2017.
[23]
N. Bekiaris-Liberis, C. Roncoli, and M. Papageorgiou, “Highway
traffic state estimation per lane in the presence of connected vehicles,
Transportation Research Part B: Methodological, vol. 106, pp. 1–28,
Dec. 2017.
[24]
S. Papadopoulou, C. Roncoli, N. Bekiaris-Liberis, I. Papamichail, and
M. Papageorgiou, “Microscopic simulation-based validation of a per-lane
traffic state estimation scheme for highways with connected vehicles,
Transportation Research Part C: Emerging Technologies, vol. 86, pp.
441–452, Jan. 2018.
[25]
M. Treiber, A. Kesting, and R. E. Wilson, “Reconstructing the traffic
state by fusion of heterogeneous data,” Computer-Aided Civil and
Infrastructure Engineering, vol. 26, no. 6, pp. 408–419, 2011.
[26]
X. Chen, S. Zhang, L. Li, and L. Li, “Adaptive rolling smoothing with
heterogeneous data for traffic state estimation and prediction, IEEE
transactions on intelligent transportation systems, vol. 20, no. 4, pp.
1247–1258, 2018.
[27]
F. Rempe, P. Franeck, U. Fastenrath, and K. Bogenberger, A phase-based
smoothing method for accurate traffic speed estimation with floating car
data,” Transportation Research Part C: Emerging Technologies, vol. 85,
pp. 644–663, 2017.
[28]
Y. Yuan, Z. Zhang, X. T. Yang, and S. Zhe, “Macroscopic traffic flow
modeling with physics regularized gaussian process: A new insight into
machine learning applications in transportation,” Transportation Research
Part B: Methodological, vol. 146, pp. 88–110, 2021.
[29]
F. Wu, Z. Cheng, H. Chen, T. Z. Qiu, and L. Sun, “Traffic state estimation
with anisotropic gaussian processes from vehicle trajectories,” arXiv
preprint arXiv:2303.02311, 2023.
[30]
F. Rempe, P. Franeck, and K. Bogenberger, “On the estimation of traffic
speeds with Deep Convolutional Neural Networks given probe data,”
Transportation research part C: emerging technologies, vol. 134, p.
103448, 2022.
[31]
W. Lu, Y. Rui, and B. Ran, “Lane-level traffic speed forecasting: a
novel mixed deep learning model, IEEE Transactions on Intelligent
Transportation Systems, vol. 23, no. 4, pp. 3601–3612, 2020.
[32]
Y. Wu, D. Zhuang, A. Labbe, and L. Sun, “Inductive graph neural
networks for spatiotemporal kriging,” in Proceedings of the AAAI
Conference on Artificial Intelligence, vol. 35, no. 5, 2021, pp. 4478–4485.
[33]
T. Nie, G. Qin, Y. Wang, and J. Sun, “Towards better traffic volume
estimation: Tackling both underdetermined and non-equilibrium problems
via a correlation adaptive graph convolution network,” arXiv preprint
arXiv:2303.05660, 2023.
[34]
K. Zhang, X. Feng, N. Jia, L. Zhao, and Z. He, “Tsr-gan: Generative
adversarial networks for traffic state reconstruction with time space
diagrams,” Physica A: Statistical Mechanics and its Applications, vol.
591, p. 126788, 2022.
[35]
D. Xu, C. Wei, P. Peng, Q. Xuan, and H. Guo, “Ge-gan: A novel deep
learning framework for road traffic state estimation, Transportation
Research Part C: Emerging Technologies, vol. 117, p. 102635, 2020.
[36]
J. Yu, M. E. Stettler, P. Angeloudis, S. Hu, and X. M. Chen, “Urban
network-wide traffic speed estimation with massive ride-sourcing gps
traces,” Transportation Research Part C: Emerging Technologies, vol.
112, pp. 136–152, 2020.
[37]
T. Nie, G. Qin, Y. Wang, and J. Sun, “Correlating sparse sensing for
network-wide traffic speed estimation: An integrated graph tensor-based
kriging approach,” arXiv preprint arXiv:2210.11780, 2022.
[38]
J. Huang and S. Agarwal, “Physics informed deep learning for traffic state
estimation,” in 2020 IEEE 23rd International Conference on Intelligent
Transportation Systems (ITSC). IEEE, 2020, pp. 1–6.
[39]
R. Shi, Z. Mo, K. Huang, X. Di, and Q. Du, “A physics-informed deep
learning paradigm for traffic state and fundamental diagram estimation,
IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 8,
pp. 11 688–11 698, 2021.
[40]
R. Shi, Z. Mo, and X. Di, “Physics-informed deep learning for traffic
state estimation: A hybrid paradigm informed by second-order traffic
models,” in Proceedings of the AAAI Conference on Artificial Intelligence,
vol. 35, no. 1, 2021, pp. 540–547.
[41]
F. Rempe, A. Loder, and K. Bogenberger, “Estimating motorway traffic
states with data fusion and physics-informed deep learning,” in 2021
IEEE International Intelligent Transportation Systems Conference (ITSC).
IEEE, 2021, pp. 2208–2214.
[42]
C. Zhao and H. Yu, “Observer-informed deep learning for traffic state
estimation with boundary sensing,” IEEE Transactions on Intelligent
Transportation Systems, 2023.
[43]
J. Zhang, S. Mao, L. Yang, W. Ma, S. Li, and Z. Gao, “Physics-informed
deep learning for traffic state estimation based on the traffic flow model
and computational graph method,” Information Fusion, vol. 101, p.
101971, 2024.
[44]
J. Lu, C. Li, X. B. Wu, and X. S. Zhou, “Physics-informed neural
networks for integrated traffic state and queue profile estimation: A
differentiable programming approach on layered computational graphs,”
Transportation Research Part C: Emerging Technologies, vol. 153, p.
104224, 2023.
[45]
W. Shao and L. Chen, “License plate recognition data-based traffic volume
estimation using collaborative tensor decomposition, IEEE Transactions
on Intelligent Transportation Systems, vol. 19, no. 11, pp. 3439–3448,
2018.
[46]
K. Tang, C. Tan, Y. Cao, J. Yao, and J. Sun, “A tensor decomposition
method for cycle-based traffic volume estimation using sampled vehicle
trajectories,” Transportation research part C: emerging technologies, vol.
118, p. 102739, 2020.
[47]
T. Seo, T. Kusakabe, and Y. Asakura, “Estimation of flow and
density using probe vehicles with spacing measurement equipment,”
Transportation Research Part C: Emerging Technologies, vol. 53, pp.
134–150, 2015.
[48]
Y. Han and S. Ahn, “Estimation of traffic flow rate with data from
connected-automated vehicles using bayesian inference and deep learning,”
Frontiers in Future Transportation, vol. 2, p. 644988, 2021.
[49]
V. Kyriacou, Y. Englezou, C. G. Panayiotou, and S. Timotheou,
“Bayesian traffic state estimation using extended floating car data, IEEE
Transactions on Intelligent Transportation Systems, vol. 24, no. 2, pp.
1518–1532, 2022.
[50]
R. Ke, Z. Li, S. Kim, J. Ash, Z. Cui, and Y. Wang, “Real-time bidirectional
traffic flow parameter estimation from aerial videos, IEEE Transactions
on Intelligent Transportation Systems, vol. 18, no. 4, pp. 890–901, 2016.
[51]
K. Theocharides, C. Menelaou, Y. Englezou, and S. Timotheou, “Towards
efficient traffic state estimation using sparse uav-based data in urban
networks,” in 2023 31st Mediterranean Conference on Control and
14
Automation (MED). IEEE, 2023, pp. 1–6.
[52]
R. Ke, Z. Li, J. Tang, Z. Pan, and Y. Wang, “Real-time traffic flow
parameter estimation from uav video based on ensemble classifier and
optical flow, IEEE Transactions on Intelligent Transportation Systems,
vol. 20, no. 1, pp. 54–64, 2018.
[53]
K. Theocharides, C. Menelaou, Y. Englezou, and S. Timotheou,
“Real-time unmanned aerial vehicle-based traffic state estimation for
multi-regional traffic networks, Transportation Research Record, p.
03611981231213079, 2024.
[54]
G. F. Newell, “A simplified theory of kinematic waves in highway traffic,
part i: General theory, Transportation Research Part B: Methodological,
vol. 27, no. 4, pp. 281–287, 1993.
[55]
J. A. Laval, “Hysteresis in traffic flow revisited: An improved
measurement method,” Transportation Research Part B: Methodological,
vol. 45, no. 2, pp. 385–391, 2011.
[56]
N. Tsanakas, J. Ekstr
¨
om, and J. Olstam, “Generating virtual vehicle
trajectories for the estimation of emissions and fuel consumption,”
Transportation Research Part C: Emerging Technologies, vol. 138, p.
103615, 2022.
[57]
W. Ma and S. Qian, “High-resolution traffic sensing with probe
autonomous vehicles: A data-driven approach, Sensors, vol. 21, no. 2,
p. 464, 2021.
[58]
Z. He, Y. Lv, L. Lu, and W. Guan, “Constructing spatiotemporal speed
contour diagrams: using rectangular or non-rectangular parallelogram
cells?” Transportmetrica B: transport dynamics, vol. 7, no. 1, pp. 44–60,
2019.
[59]
X. Wang and L. Sun, “Diagnosing spatiotemporal traffic anomalies
with low-rank tensor autoregression, IEEE Transactions on Intelligent
Transportation Systems, vol. 22, no. 12, pp. 7904–7913, 2021.
[60]
D. Chen, S. Ahn, J. Laval, and Z. Zheng, “On the periodicity of
traffic oscillations and capacity drop: the role of driver characteristics,
Transportation research part B: methodological, vol. 59, pp. 117–136,
2014.
[61]
Y. Hu, D. Zhang, J. Ye, X. Li, and X. He, “Fast and accurate matrix
completion via truncated nuclear norm regularization,” IEEE transactions
on pattern analysis and machine intelligence, vol. 35, no. 9, pp. 2117–
2130, 2012.
[62]
B. Karl, L. Kessler, and K. Bogenberger, “Automated classification of
different congestion types,” in 2019 IEEE Intelligent Transportation
Systems Conference (ITSC). IEEE, 2019, pp. 2312–2317.
[63]
E. J. Cand
`
es, X. Li, Y. Ma, and J. Wright, “Robust principal component
analysis?” Journal of the ACM (JACM), vol. 58, no. 3, pp. 1–37, 2011.
[64]
M. J. Lighthill and G. B. Whitham, “On kinematic waves ii. a theory of
traffic flow on long crowded roads, Proceedings of the Royal Society
of London. Series A. Mathematical and Physical Sciences, vol. 229, no.
1178, pp. 317–345, 1955.
[65]
P. I. Richards, “Shock waves on the highway, Operations research,
vol. 4, no. 1, pp. 42–51, 1956.
[66]
J. Van Lint and S. P. Hoogendoorn, A robust and efficient method for
fusing heterogeneous data from traffic sensors on freeways, Computer-
Aided Civil and Infrastructure Engineering, vol. 25, no. 8, pp. 596–612,
2010.
[67]
M. Treiber and D. Helbing, “Reconstructing the spatio-temporal traffic
dynamics from stationary detector data,” Cooperative Transportation
Dynamics, no. 1, pp. 3.1–3.24, 2002.
[68]
B. S. Kerner, Introduction to modern traffic flow theory and control: the
long road to three-phase traffic theory. Springer Science & Business
Media, 2009.
[69]
X. Chen, J. Yin, G. Qin, K. Tang, Y. Wang, and J. Sun, “Integrated
macro-micro modelling for individual vehicle trajectory reconstruction
using fixed and mobile sensor data,” Transportation Research Part C:
Emerging Technologies, vol. 145, p. 103929, 2022.
[70]
Z. Chen, C. Yang, and A. Chen, “Estimating fuel consumption and
emissions based on reconstructed vehicle trajectories,” Journal of
Advanced Transportation, vol. 48, no. 6, pp. 627–641, 2014.
Yang He received his B.E. degree in Traffic Engi-
neering from Chang’ an University, Xi’an, China, in
2020. He is currently pursuing the Ph.D. degree in the
Intelligent Transportation System Research Center at
Southeast University. His current research interests
include traffic state estimation, network modeling,
and low-rank modeling.
Chengchuan An received his Ph.D. degree in Trans-
portation Engineering from Southeast University,
Nanjing, China, in 2019. From 2014 to 2016, he
was a visiting scholar at the Department of Civil
and Architectural Engineering and Mechanics, Uni-
versity of Arizona, USA. Since 2020, he has been a
post-doctor in the Intelligent Transportation System
Research Center at Southeast University. His current
research interests include intelligent traffic signal
control systems and traffic data mining.
Yuheng Jia received the B.S. degree in automation
and the M.S. degree in control theory and engineering
from Zhengzhou University, Zhengzhou, China, in
2012 and 2015, respectively, and the Ph.D. degree in
computer science from the City University of Hong
Kong, SAR, China, in 2019. He is currently an asso-
ciate professor with the School of Computer Science
and Engineering, Southeast University, China. His
research interests include machine learning, Bayesian
method, spectral clustering and low-rank modeling.
Jiachao Liu received his B.S. degree in Traffic
Engineering from Dalian University of Technology,
Dalian, China, in 2017, M.S. Degree in Transporta-
tion Engineering from Southeast University, Nanjing,
China, in 2020, and M.S. Degree in Machine Learn-
ing from Carnegie Mellon University, USA, in 2023.
He is currently pursuing the Ph.D. degree in Civil
and Environmental Engineering at Carnegie Mellon
University, USA. His research interests include trans-
portation network modeling, simulation, optimization
and machine learning.
Zhenbo Lu received the Ph.D. degree in Traffic
Information Engineering and Control from Southeast
University, Nanjing, China, in 2011. He is currently
an Associate Professor with the Intelligent Transporta-
tion System Research Center, Southeast University.
His main research interests include transportation
planning, traffic simulation, and intelligent transporta-
tion systems.
Jingxin Xia is a professor at the Intelligent Trans-
portation System Research Center, Southeast Univer-
sity, Nanjing, China. He received the Ph.D. degree
in Transportation Engineering from the University
of Kentucky, USA in 2006. He has published more
than forty peer-reviewed papers so far, and his main
research interests include traffic flow theory, trans-
portation network modeling, traffic signal control,
and intelligent transportation systems.
... Despite their simplicity, modelbased methods are often limited by the inherent constraints of traffic flow models. These methods re-quire extensive data and time-consuming calibration of parameters, which can be a labor-intensive process (He et al., 2024). ...
Conference Paper
Traffic flow data is essential for urban planning, logistics, transport management, and similar applications. However, achieving full sensor coverage across a road network is often infeasible due to high installation and maintenance costs. Simulation data from traffic models can help in filling this gap. However, calibrating and validating these traffic models is time-consuming. This paper presents a framework that combines real-time traffic flow predictions from sensor-equipped road segments with 24-hour static simulation data across an entire network. By applying a method based on the Breadth-First Search algorithm, this framework updates network-wide traffic flow by utilizing the data-driven predictions at sensor-equipped road segments and simulation data. Evaluation on a network with over 27000 road segments shows that this approach improves prediction accuracy over static simulation and is viable for real-time deployment.
Article
Full-text available
Accurately monitoring road traffic state is crucial for various applications, including travel time prediction, traffic control, and traffic safety. However, the lack of sensors often results in incomplete traffic state data, making it challenging to obtain reliable information for decision-making. This paper proposes a novel method for imputing traffic state data using Gaussian processes (GP) to address this issue. We propose a kernel rotation re-parametrization scheme that transforms a standard isotropic GP kernel into an anisotropic kernel, which can better model the congestion propagation in traffic flow data. The model parameters can be estimated by statistical inference using data from sparse probe vehicles or loop detectors. Moreover, the rotated GP method provides statistical uncertainty quantification for the imputed traffic state, making it more reliable. We also extend our approach to a multi-output GP, which allows for simultaneously estimating the traffic state for multiple lanes. We evaluate our method using real-world traffic data from the Next Generation simulation (NGSIM) and HighD programs, along with simulated data representing a traffic bottleneck scenario. Considering current and future mixed traffic of connected vehicles (CVs) and human-driven vehicles (HVs), we experiment with the traffic state estimation (TSE) scheme from 5% to 50% available trajectories, mimicking different CV penetration rates in a mixed traffic environment. We also test the traffic state estimation when traffic flow information is obtained from loop detectors. The results demonstrate the adaptability of our TSE method across different CV penetration rates and types of detectors, achieving state-of-the-art accuracy in scenarios with sparse observations.
Article
Traffic state estimation is a challenging task because of the collection of sparse and noisy measurements from fixed points in the traffic network. Induction loops, as they are non-intrusive, can observe any area of the traffic network on demand and provide accurate traffic density and speed measurements. Our main contribution is the development of an optimization framework where small parts of the traffic network are monitored by Unmanned Aerial Vehicles (UAVs) and accurate estimates of traffic density and mean speeds for every region in the traffic network are returned in real-time. Assuming regional-based traffic dynamics, a cyclical UAV flight path is defined for each region. One UAV is assigned to each flight path and monitors a small area of the region below. The UAV-based traffic measurements are expressed as moving averages to smooth out fluctuations in traffic density and mean speed. A moving horizon optimization problem is formulated, which minimizes the estimation and process errors over a moving time window. The problem is non-convex and challenging to solve, because of the presence of nonlinear traffic dynamics. By considering free-flow conditions, the optimization problem is recast to a quadratic program that returns density estimations for each region of the traffic network in real-time. Simulation results compare our UAV framework to an alternative, where the whole traffic network is monitored by UAVs. Both frameworks obtain similar results, despite the alternative framework using more UAVs than our framework.
Article
Traffic state estimation (TSE) refers to the inference of macroscopic traffic states, including density, speed, and flow, based on partially observed traffic data and some prior knowledge of traffic dynamics. TSE plays a key role in traffic management since traffic control relies on accurate estimation of traffic states. This paper proposes a novel hybrid TSE approach called Observer-Informed Deep Learning (OIDL), which integrates a Partial Differential Equation (PDE) observer and deep learning paradigm to estimate spatial-temporal traffic states from boundary sensing data. The proposed OIDL consists of two modules, an Observer-Uninformed Neural Network (OUNN) to generate preliminary traffic state estimation, and an Observer-Informed Neural Network (OINN) constructed from a boundary observer with theoretical convergence guarantee to regularize the estimation. Furthermore, we propose Adaptive OIDL (aOIDL) to simultaneously estimate traffic states and model parameters. Experiments on the NGSIM dataset demonstrate that the proposed OIDL reduces the estimation error by up to 30 percent compared to the model-based observer, data-driven neural networks, and some hybrid TSE approaches. The OIDL also has smaller variance of the estimation error and presents more accurate pattern for congested traffic.
Article
This paper studies the traffic state estimation (TSE) problem using sparse observations from mobile sensors. Most existing TSE methods either rely on well-defined physical traffic flow models or require large amounts of simulation data as input to train learning algorithms. Different from previous studies, in this paper we propose a purely data-driven and model-free solution. We consider TSE as a spatiotemporal matrix completion/interpolation problem and apply spatiotemporal delay embedding to transform the original incomplete matrix into a fourth-order Hankel structured tensor. By imposing a low-rank assumption on this tensor structure, we can approximate and characterize both global patterns and local patterns in a data-driven manner. We use a truncated nuclear norm of a balanced spatiotemporal unfolding to approximate the tensor rank and develop an efficient algorithm based on the Alternating Direction Method of Multipliers (ADMM) to solve the problem. The proposed framework only involves two hyperparameters, spatial and temporal window lengths, which are easy to set given the degree of data sparsity. To validate the effectiveness of our proposed method, we conducted numerical experiments on real-world high-resolution trajectory data, which demonstrated its superiority in some challenging scenarios. The proposed method shows great potential for solving the TSE problem using sparse observations from mobile sensors and can be applied in various traffic applications.
Article
Traffic state estimation is a challenging task due to the collection of sparse and noisy measurements from specific points of the traffic network. The emergence of Connected and Automated Vehicles (CAVs) provides new capabilities for traffic state estimation using extended floating car data such as position, speed and spacing information. In this work we propose a Bayesian Traffic State Estimation (BTSE) methodology for estimating the traffic density based on extended floating car data. BTSE utilizes the Bayesian paradigm to express any prior information to derive probability distributions of the traffic density of different road segments of the traffic network. Two variations of the BTSE methodology are developed to handle the offline and online estimation problem. The BTSE methodology is evaluated both using realistic SUMO micro-simulations for M25 Highway, London, U.K., and a real-life vehicle-trajectory dataset from German highways, extracted from videos recorded by drones. The efficiency and accuracy of the BTSE methodology is compared to an existing methodology in the literature. We present results for the estimation performance of the methods showing that the Bayesian methodology consistently results in lower mean absolute percentage error than the compared literature method. The BTSE methodology yields high-quality estimation results even for a low penetration rate of CAVs (e.g. 5%).