Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping

Abstract

Developing agents that can execute multiple skills by learning from pre-collected datasets is an important problem in robotics, where online interaction with the environment is extremely time-consuming. Moreover, manually designing reward functions for every single desired skill is prohibitive. Prior works targeted these challenges by learning goal-conditioned policies from offline datasets without manually specified rewards, through hindsight relabeling. These methods suffer from the issue of sparsity of rewards, and fail at long-horizon tasks. In this work, we propose a novel self-supervised learning phase on the pre-collected dataset to understand the structure and the dynamics of the model, and shape a dense reward function for learning policies offline. We evaluate our method on three continuous control tasks, and show that our model significantly outperforms existing approaches, especially on tasks that involve long-term planning.
Fichier principal
Vignette du fichier
offline_reward_shaping_CoRL22.pdf (2.31 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03869706 , version 1 (24-11-2022)

Identifiers

  • HAL Id : hal-03869706 , version 1

Cite

Lina Mezghani, Sainbayar Sukhbaatar, Piotr Bojanowski, Alessandro Lazaric, Karteek Alahari. Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping. CoRL 2022- Conference on Robot Learning, Dec 2022, Auckland, New Zealand. pp.1-15. ⟨hal-03869706⟩
64 View
129 Download

Share

Gmail Facebook X LinkedIn More