Reconciling Spatial and Temporal Abstractions for Goal Representation - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2024

Reconciling Spatial and Temporal Abstractions for Goal Representation

Résumé

Goal representation affects the performance of Hierarchical Reinforcement Learning (HRL) algorithms by decomposing the complex learning problem into easier subtasks. Recent studies show that representations that preserve temporally abstract environment dynamics are successful in solving difficult problems and provide theoretical guarantees for optimality. These methods however cannot scale to tasks where environment dynamics increase in complexity i.e. the temporally abstract transition relations depend on larger number of variables. On the other hand, other efforts have tried to use spatial abstraction to mitigate the previous issues. Their limitations include scalability to high dimensional environments and dependency on prior knowledge. In this paper, we propose a novel three-layer HRL algorithm that introduces, at different levels of the hierarchy, both a spatial and a temporal goal abstraction. We provide a theoretical study of the regret bounds of the learned policies. We evaluate the approach on complex continuous control tasks, demonstrating the effectiveness of spatial and temporal abstractions learned by this approach.
Fichier principal
Vignette du fichier
Zadem2024TICLR.pdf (1.68 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04629549 , version 1 (29-06-2024)

Licence

Identifiants

Citer

Mehdi Zadem, Sergio Mover, Sergio Mover, Sao Mai Nguyen. Reconciling Spatial and Temporal Abstractions for Goal Representation. ICRL 2024 - The Twelfth International Conference on Learning Representations, May 2024, Vienna, Austria. ⟨hal-04629549⟩
0 Consultations
0 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More