Think Before You Act: Unified Policy for Interleaving Language Reasoning with Actions - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Think Before You Act: Unified Policy for Interleaving Language Reasoning with Actions

Résumé

The success of transformer models trained with a language modeling objective brings a promising opportunity to the reinforcement learning framework. Decision Transformer is a step towards this direction, showing how to train transformers with a similar next-step prediction objective on offline data. Another important development in this area is the recent emergence of large-scale datasets collected from the internet, such as the ones composed of tutorial videos with captions where people talk about what they are doing. To take advantage of this language component, we propose a novel method for unifying language reasoning with actions in a single policy. Specifically, we augment a transformer policy with word outputs, so it can generate textual captions interleaved with actions. When tested on the most challenging task in BabyAI, with captions describing next subgoals, our reasoning policy consistently outperforms the caption-free baseline.
Fichier principal
Vignette du fichier
mezghani.pdf (896.84 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04107094 , version 1 (25-05-2023)

Licence

Paternité

Identifiants

Citer

Lina Mezghani, Piotr Bojanowski, Karteek Alahari, Sainbayar Sukhbaatar. Think Before You Act: Unified Policy for Interleaving Language Reasoning with Actions. RRL 2023 - Workshop on Reincarnating Reinforcement Learning, May 2023, Kigali, Rwanda. pp.1-9. ⟨hal-04107094⟩
58 Consultations
25 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More