Predicting Future Instance Segmentations by Forecasting Convolutional Features - Inria - Institut national de recherche en sciences et technologies du numérique
Communication Dans Un Congrès Année : 2018

Predicting Future Instance Segmentations by Forecasting Convolutional Features

Résumé

Anticipating future events is an important prerequisite towards intelligent behavior. Video forecasting has been studied as a proxy task towards this goal. Recent work has shown that to predict semantic segmentation of future frames, forecasting at the semantic level is more effective than forecasting RGB frames and then segmenting these. In this paper we consider the more challenging problem of future instance seg-mentation, which additionally segments out individual objects. To deal with a varying number of output labels per image, we develop a pre-dictive model in the space of fixed-sized convolutional features of the Mask R-CNN instance segmentation model. We apply the " detection head " of Mask R-CNN on the predicted features to produce the instance segmentation of future frames. Experiments show that this approach significantly improves over baselines based on optical flow.
Fichier principal
Vignette du fichier
luc2018instpred.pdf (2.85 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01757669 , version 1 (03-04-2018)
hal-01757669 , version 2 (03-10-2018)

Identifiants

  • HAL Id : hal-01757669 , version 1

Citer

Pauline Luc, Camille Couprie, Yann Lecun, Jakob Verbeek. Predicting Future Instance Segmentations by Forecasting Convolutional Features. ECCV - European Conference on Computer Vision, 2018, Munich, Germany. ⟨hal-01757669v1⟩
952 Consultations
1302 Téléchargements

Partager

More