Online Spatio-Temporal 3D Convolutional Neural Network for Early Recognition of Handwritten Gestures - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

Online Spatio-Temporal 3D Convolutional Neural Network for Early Recognition of Handwritten Gestures

Abstract

Inspired by recent spatio-temporal Convolutional Neural Networks in computer vision field, we propose OLT-C3D (Online Long-Term Convolutional 3D), a new architecture based on a 3D Convolutional Neural Network (3D CNN) to address the complex task of early recognition of 2D handwritten gestures in real time. The input signal of the gesture is translated into an image sequence along time with the trajectory history. The image sequence is passed into our 3D CNN OLT-C3D which gives a prediction at each new frame. OLT-C3D is coupled with an integrated temporal reject system to postpone the decision in time if more information is needed. Moreover our system is end-to-end trainable, OLT-C3D and the temporal reject system are jointly trained to optimize the earliness of the decision. Our approach achieves superior performances on two complementary and freely available datasets: ILGDB and MTGSetB.
Fichier principal
Vignette du fichier
ICDAR_2021__early_reco2D_VCR.pdf (3.74 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03229957 , version 1 (19-05-2021)

Identifiers

  • HAL Id : hal-03229957 , version 1

Cite

William Mocaër, Eric Anquetil, Richard Kulpa. Online Spatio-Temporal 3D Convolutional Neural Network for Early Recognition of Handwritten Gestures. ICDAR 2021 - 16th International Conference on Document Analysis and Recognition, Sep 2021, Lausanne, Switzerland. pp.221-236. ⟨hal-03229957⟩
399 View
116 Download

Share

Gmail Mastodon Facebook X LinkedIn More