TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL


Training autonomous agents able to generalize to multiple tasks is a key target of Deep Reinforcement Learning (DRL) research. In parallel to improving DRL algorithms themselves, Automatic Curriculum Learning (ACL) study how teacher algorithms can train DRL agents more efficiently by adapting task selection to their evolving abilities. While multiple standard benchmarks exist to compare DRL agents, there is currently no such thing for ACL algorithms. Thus, comparing existing approaches is difficult, as too many experimental parameters differ from paper to paper. In this work, we identify several key challenges faced by ACL algorithms. Based on these, we present TeachMyAgent (TA), a benchmark of current ACL algorithms leveraging procedural task generation. It includes 1) challengespecific unit-tests using variants of a procedural Box2D bipedal walker environment, and 2) a new procedural Parkour environment combining most ACL challenges, making it ideal for global performance assessment. We then use TeachMyAgent to conduct a comparative study of representative existing approaches, showcasing the competitiveness of some ACL algorithms that do not use expert knowledge. We also show that the Parkour environment remains an open problem. We open-source our environments, all studied ACL algorithms (collected from open-source code or re-implemented), and DRL students in a Python package available at https://github. com/flowersteam/TeachMyAgent.
Fichier principal
Vignette du fichier
TeachMyAgent.pdf (6.71 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03173198 , version 1 (18-03-2021)
hal-03173198 , version 2 (09-06-2021)



Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer. TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL. ICML 2021 - Thirty-eighth International Conference on Machine Learning, Jul 2021, Vienna / Virtual, Austria. pp.9052--9063. ⟨hal-03173198v2⟩
141 View
32 Download



Gmail Facebook X LinkedIn More