Reference interpretation in a multimodal environment combining speech and gesture - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 1995

Reference interpretation in a multimodal environment combining speech and gesture

Nadia Bellalem
  • Fonction : Auteur
Laurent Romary

Résumé

The study presented in this paper is dedicated to the integration of pointing gestures within a task oriented man-machine dialogue system. From our point of view, it is natural to see this problem as a sub-part of the more general study of reference in a dialogue since we took the option to limit the analysis of gestures to those explicitly associated with speech and more specifically for singling out objects in the task. Even under these hypotheses, understanding the global message resulting from the combinaison of speech and gesture implies that a precise analysis of the gestural trajectory should be done. This analysis may be split up into two main steps. The first, which we may call structural, is centred upon the shape of the gestural trajectory to mark the meaningful parts to which a specific designation role will be given. The second step has to do with the contextual interpretation of the gesture for which one has to take into account a) the features of the application and the way it is visualized and b) the oral dialogue and more specifically the instructional content of the referring expression accompanying the gesture.
Fichier principal
Vignette du fichier
ReferenceInterpretation.pdf (117.36 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-00521585 , version 1 (05-08-2014)

Identifiants

  • HAL Id : hal-00521585 , version 1

Citer

Nadia Bellalem, Laurent Romary. Reference interpretation in a multimodal environment combining speech and gesture. First International Workshop on Intelligence and Multimodality in Multimedia Interfaces, 1995, Edinburgh, United Kingdom. 5 p. ⟨hal-00521585⟩

Collections

CNRS UNIV-LORRAINE
226 Consultations
51 Téléchargements

Partager

Gmail Facebook X LinkedIn More