Is attention to bounding boxes all you need for pedestrian action prediction? - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2021

Is attention to bounding boxes all you need for pedestrian action prediction?

Abstract

The human driver is no longer the only one concerned with the complexity of the driving scenarios. Autonomous vehicles (AV) are similarly becoming involved in the process. Nowadays, the development of AV in urban places underpins essential safety concerns for vulnerable road users (VRUs) such as pedestrians. Therefore, to make the roads safer, it is critical to classify and predict their future behavior. In this paper, we present a framework based on multiple variations of the Transformer models to reason attentively about the dynamic evolution of the pedestrians' past trajectory and predict its future actions of crossing or not crossing the street. We proved that using only bounding-boxes as input to our model can outperform the previous state-of-the-art models and reach a prediction accuracy of 91% and an F1-score of 0.83 on the PIE dataset up to two seconds ahead in the future. In addition, we introduced a large-size simulated dataset (CP2A) using CARLA for action prediction. Our model has similarly reached high accuracy (91 %) and F1-score (0.91) on this dataset. Interestingly, we showed that pre-training our Transformer model on the simulated dataset and then fine-tuning it on the real dataset can be very effective for the action prediction task.
Fichier principal
Vignette du fichier
TransBB.pdf (2.54 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03288986 , version 1 (16-07-2021)

Identifiers

  • HAL Id : hal-03288986 , version 1

Cite

Lina Achaji, Julien Moreau, Thibault Fouqueray, Francois Aioun, François Charpillet. Is attention to bounding boxes all you need for pedestrian action prediction?. 2021. ⟨hal-03288986⟩
97 View
225 Download

Share

Gmail Facebook X LinkedIn More