Human Trajectory Prediction via Neural Social Physics

Abstract

Trajectory prediction has been widely pursued in many fields, and many model-based and model-free methods have been explored. The former include rule-based, geometric or optimization-based models, and the latter are mainly comprised of deep learning approaches. In this paper, we propose a new method combining both methodologies based on a new Neural Differential Equation model. Our new model (Neural Social Physics or NSP) is a deep neural network within which we use an explicit physics model with learnable parameters. The explicit physics model serves as a strong inductive bias in modeling pedestrian behaviors, while the rest of the network provides a strong data-fitting capability in terms of system parameter estimation and dynamics stochasticity modeling. We compare NSP with 15 recent deep learning methods on 6 datasets and improve the state-of-the-art performance by 5.56%-70%. Besides, we show that NSP has better generalizability in predicting plausible trajectories in drastically different scenarios where the density is 2-5 times as high as the testing data. Finally, we show that the physics model in NSP can provide plausible explanations for pedestrian behaviors, as opposed to black-box deep learning. Code is available: https://github.com/realcrane/Human-Trajectory-Prediction-via-Neural-Social-Physics.

Full text

Human Trajectory Prediction via Neural Social Physics
Jiangbei Yue, Dinesh Manocha, and He Wang
Motivation
Existing approaches generally fall into model-based and model-free methods. Model-
based methods tend to possess good explainability. However, they are less effective in
data fitting. Model-free methods based on deep learning excel at data fitting, but lack
explainability. Our paper proposes neural social physics that can explain pedestrian
behaviors and retain good data-fitting capabilities to predict human trajectories by
combining model-based and model-free methods.
Contributions
•A new neural differentiable equation model for trajectory prediction and analysis.
•A new mechanism to combine explicit models with neural networks for prediction.
•The NSP model performs well in: prediction, generalization and explainability
NSP
𝒅𝒒
𝒅𝒕 𝒕 = 𝒇𝜽,𝝓 𝒕, 𝒒 𝒕 , 𝜴 𝒕 , 𝒒𝑻, 𝑬 + 𝜶𝝓(𝒕, 𝒒𝒕:𝒕−𝑴)
𝒒 𝒕 + 𝜟𝒕 ≈ 𝒒 𝒕 + ሶ
𝒒 𝒕 ∆𝒕 = 𝒑 𝒕
ሶ
𝒑 𝒕 + ∆𝒕 ሶ
𝒑 𝒕 + 𝜶(𝒕, 𝒒𝒕:𝒕−𝑴 )
ሷ
𝑷(𝒕)
NSP-SFM
Experiments
Dataset S-GAN Sophie PECNet Y-net NSP
ETH 0.81/1.52 0.70/1.43 0.54/0.87 0.28/0.33 0.25/0.24
Hotel 0.72/1.61 0.76/1.67 0.18/0.24 0.10/0.14 0.09/0.13
Univ 0.60/1.26 0.54/1.24 0.35/0.60 0.24/0.41 0.21/0.38
Zara1 0.34/0.69 0.30/0.63 0.22/0.39 0.17/0.27 0.16/0.27
Zara2 0.42/0.84 0.38/0.78 0.17/0.30 0.13/0.22 0.12/0.20
AVG 0.58/1.18 0.54/1.15 0.29/0.48 0.18/0.27 0.17/0.24
SDD 27.2/41.4 16.3/29.4 10.0/15.9 7.9/11.9 6.5/10.6
Prediction accuracy on public datasets. ADE/FDE
Interpretability of Prediction
Collisions in unseen scenarios
Red is observed. Green is our prediction. Black is the ground-truth. Blue is
pedestrians. 𝐹
𝑔𝑜𝑎𝑙,𝐹𝑐𝑜𝑙 and 𝐹
𝑒𝑛𝑣 are shown as yellow, blue and black arrows.