Memory-Augmented Reinforcement Learning for Image-Goal Navigation - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Memory-Augmented Reinforcement Learning for Image-Goal Navigation


In this work, we present a memory-augmented approach for image-goal navigation. Earlier attempts, including RL-based and SLAM-based approaches have either shown poor generalization performance, or are heavily-reliant on pose/depth sensors. Our method is based on an attention-based end-to-end model that leverages an episodic memory to learn to navigate. First, we train a state-embedding network in a selfsupervised fashion, and then use it to embed previously-visited states into the agent’s memory. Our navigation policy takes advantage of this information through an attention mechanism. We validate our approach with extensive evaluations, and show that our model establishes a new state of the art on the challenging Gibson dataset. Furthermore, we achieve this impressive performance from RGB input alone, without access to additional information such as position or depth, in stark contrast to related work.
Fichier principal
Vignette du fichier
MemAugNav_IROS.pdf (2.62 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03110875 , version 1 (14-01-2021)
hal-03110875 , version 2 (02-03-2022)
hal-03110875 , version 3 (12-09-2022)



Lina Mezghani, Sainbayar Sukhbaatar, Thibaut Lavril, Oleksandr Maksymets, Dhruv Batra, et al.. Memory-Augmented Reinforcement Learning for Image-Goal Navigation. IROS - IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct 2022, Kyoto, Japan. ⟨hal-03110875v3⟩
304 View
340 Download



Gmail Mastodon Facebook X LinkedIn More