A Multimodal Human-Robot Interaction Dataset - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Poster Communications Year : 2016

A Multimodal Human-Robot Interaction Dataset


This works presents a multimodal dataset for Human-Robot Interactive Learning. 1 The dataset contains synchronized recordings of several human users, from a stereo 2 microphone and three cameras mounted on the robot. The focus of the dataset is 3 incremental object learning, oriented to human-robot assistance and interaction. To 4 learn new object models from interactions with a human user, the robot needs to 5 be able to perform multiple tasks: (a) recognize the type of interaction (pointing, 6 showing or speaking), (b) segment regions of interest from acquired data (hands and 7 objects), and (c) learn and recognize object models. We illustrate the advantages 8 of multimodal data over camera-only datasets by presenting an approach that 9 recognizes the user interaction by combining simple image and language features.
Fichier principal
Vignette du fichier
multimodal-dataset-nips(1).pdf (6.23 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-01402479 , version 1 (07-12-2016)


  • HAL Id : hal-01402479 , version 1


Pablo Azagra, Yoan Mollard, Florian Golemo, Ana Cristina Murillo, Manuel Lopes, et al.. A Multimodal Human-Robot Interaction Dataset. NIPS 2016, workshop Future of Interactive Learning Machines, Dec 2016, Barcelona, Spain. ⟨hal-01402479⟩
535 View
757 Download


Gmail Facebook X LinkedIn More