FPVRGame: Deep Learning for Hand Pose Recognition in Real-Time Using Low-End HMD - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2019

FPVRGame: Deep Learning for Hand Pose Recognition in Real-Time Using Low-End HMD

Résumé

Head Mounted Display (HMD) became a popular device, drastically increasing the usage of Virtual, Mixed, and Augmented Reality. While the systems’ visual resources are accurate and immersive, precise interfaces require depth cameras or special joysticks, requiring either complex devices or not following the natural body expression. This work presents an approach for the usage of bare hands to control an immersive game from an egocentric perspective and built from a proposed case study methodology. We used a DenseNet Convolutional Neural Network (CNN) architecture to perform the recognition in real-time, from both indoor and outdoor environments, not requiring any image segmentation process. Our research also generated a vocabulary, considering users’ preferences, seeking a set of natural and comfortable hand poses and evaluated users’ satisfaction and performance for an entertainment setup. Our recognition model achieved an accuracy of 97.89%. The user’s studies show that our method outperforms the classical controllers in regards to natural interactions. We demonstrate our results using commercial low-end HMD’s and compare our solution with state-of-the-art methods.
Fichier principal
Vignette du fichier
491829_1_En_6_Chapter.pdf (1.89 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03652048 , version 1 (26-04-2022)

Licence

Paternité

Identifiants

Citer

Eder De Oliveira, Esteban Clua, Cristina Nader Vasconcelos, Bruno Marques, Daniela Trevisan, et al.. FPVRGame: Deep Learning for Hand Pose Recognition in Real-Time Using Low-End HMD. 1st Joint International Conference on Entertainment Computing and Serious Games (ICEC-JCSG), Nov 2019, Arequipa, Peru. pp.70-84, ⟨10.1007/978-3-030-34644-7_6⟩. ⟨hal-03652048⟩
20 Consultations
55 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More