Is attention to bounding boxes all you need for pedestrian action prediction? - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2022

Is attention to bounding boxes all you need for pedestrian action prediction?

Is attention to bounding boxes all you need for pedestrian action prediction?

Résumé

The human driver is no longer the only one concerned with the complexity of the driving scenarios. Autonomous vehicles (AV) are similarly becoming involved in the process. Nowadays, the development of AV in urban places raises essential safety concerns for vulnerable road users (VRUs) such as pedestrians. Therefore, to make the roads safer, it is critical to classify and predict the pedestrians’ future behavior. In this paper, we present a framework based on multiple variations of the Transformer models able to predict the pedestrian street-crossing decision-making based on the dynamics of its initiated trajectory. We showed that using solely bounding boxes as input features can outperform the previous state-of-the-art results by reaching a prediction accuracy of 91% and an F1-score of 0.83 on the PIE dataset. In addition, we introduced a large-size simulated dataset (CP2A) using CARLA for action prediction. Our model has similarly reached high accuracy (91%) and F1-score (0.91) on this dataset. Interestingly, we showed that pre-training our Transformer model on the CP2A dataset and then fine-tuning it on the PIE dataset is beneficial for the action prediction task. Finally, our model’s results are successfully supported by the “human attention to bounding boxes” experiment which we created to test humans’ ability for pedestrian action prediction without the need for environmental context. The code for the dataset and the models is available at: https://github.com/ linaashaji/Action_Anticipation

Dates et versions

hal-03952361 , version 1 (23-01-2023)

Identifiants

Citer

Lina Achaji, Julien Moreau, Thibault Fouqueray, Francois Aioun, François Charpillet. Is attention to bounding boxes all you need for pedestrian action prediction?. 2022 IEEE Intelligent Vehicles Symposium (IV), Jun 2022, Aachen, Germany. pp.895-902, ⟨10.1109/IV51971.2022.9827084⟩. ⟨hal-03952361⟩
12 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More