Preference-Based Policy Learning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2011

Preference-Based Policy Learning


Many machine learning approaches in robotics, based on re- inforcement learning, inverse optimal control or direct policy learning, critically rely on robot simulators. This paper investigates a simulator- free direct policy learning, called Preference-based Policy Learning (PPL). PPL iterates a four-step process: the robot demonstrates a candidate pol- icy; the expert ranks this policy comparatively to other ones according to her preferences; these preferences are used to learn a policy return estimate; the robot uses the policy return estimate to build new can- didate policies, and the process is iterated until the desired behavior is obtained. PPL requires a good representation of the policy search space be available, enabling one to learn accurate policy return estimates and limiting the human ranking effort needed to yield a good policy. Further- more, this representation cannot use informed features (e.g., how far the robot is from any target) due to the simulator-free setting. As a second contribution, this paper proposes a representation based on the agnostic exploitation of the robotic log. The convergence of PPL is analytically studied and its experimental validation on two problems, involving a single robot in a maze and two interacting robots, is presented.
Fichier principal
Vignette du fichier
Preference-based_Policy_Learning.pdf (248.74 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

inria-00625001 , version 1 (20-09-2011)


  • HAL Id : inria-00625001 , version 1


Riad Akrour, Marc Schoenauer, Michèle Sebag. Preference-Based Policy Learning. European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Sep 2011, Athènes, Greece. pp.12-27. ⟨inria-00625001⟩
336 View
1839 Download


Gmail Facebook Twitter LinkedIn More