Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2018

Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics

Abstract

The most data-efficient algorithms for reinforcement learning in robotics are model-based policy search algorithms , which alternate between learning a dynamical model of the robot and optimizing a policy to maximize the expected return given the model and its uncertainties. Among the few proposed approaches, the recently introduced Black-DROPS algorithm exploits a black-box optimization algorithm to achieve both high data-efficiency and good computation times when several cores are used; nevertheless, like all model-based policy search approaches, Black-DROPS does not scale to high dimensional state/action spaces. In this paper, we introduce a new model learning procedure in Black-DROPS that leverages parameterized black-box priors to (1) scale up to high-dimensional systems, and (2) be robust to large inaccuracies of the prior information. We demonstrate the effectiveness of our approach with the " pendubot " swing-up task in simulation and with a physical hexapod robot (48D state space, 18D action space) that has to walk forward as fast as possible. The results show that our new algorithm is more data-efficient than previous model-based policy search algorithms (with and without priors) and that it can allow a physical 6-legged robot to learn new gaits in only 16 to 30 seconds of interaction time.
Fichier principal
Vignette du fichier
1709.06917.pdf (1.7 Mo) Télécharger le fichier
Origin Files produced by the author(s)
Loading...

Dates and versions

hal-01768285 , version 1 (17-04-2018)

Identifiers

Cite

Konstantinos Chatzilygeroudis, Jean-Baptiste Mouret. Using Parameterized Black-Box Priors to Scale Up Model-Based Policy Search for Robotics. ICRA 2018 - International Conference on Robotics and Automation, May 2018, brisbane, Australia. ⟨hal-01768285⟩
149 View
240 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More