Learning Reward Functions for Robotic Manipulation by Observing Humans - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Learning Reward Functions for Robotic Manipulation by Observing Humans


Observing a human demonstrator manipulate objects provides a rich, scalable and inexpensive source of data for learning robotic policies. However, transferring skills from human videos to a robotic manipulator poses several challenges, not least a difference in action and observation spaces. In this work, we use unlabeled videos of humans solving a wide range of manipulation tasks to learn a task-agnostic reward function for robotic manipulation policies. Thanks to the diversity of this training data, the learned reward function sufficiently generalizes to image observations from a previously unseen robot embodiment and environment to provide a meaningful prior for directed exploration in reinforcement learning. The learned rewards are based on distances to a goal in an embedding space learned using a time-contrastive objective. By conditioning the function on a goal image, we are able to reuse one model across a variety of tasks. Unlike prior work on leveraging human videos to teach robots, our method, Human Offline Learned Distances (HOLD) requires neither a priori data from the robot environment, nor a set of taskspecific human demonstrations, nor a predefined notion of correspondence across morphologies, yet it is able to accelerate training of several manipulation tasks on a simulated robot arm compared to using only a sparse reward obtained from task completion.
Fichier principal
Vignette du fichier
2211.09019.pdf (2.09 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03997549 , version 1 (20-02-2023)
hal-03997549 , version 2 (10-10-2023)



Minttu Alakuijala, Gabriel Dulac-Arnold, Julien Mairal, Jean Ponce, Cordelia Schmid. Learning Reward Functions for Robotic Manipulation by Observing Humans. ICRA 2023 - IEEE International Conference on Robotics and Automation, May 2023, London, United Kingdom. pp.1-11. ⟨hal-03997549v2⟩
104 View
51 Download



Gmail Facebook X LinkedIn More