Unsupervised space-time network for temporally-consistent segmentation of multiple motions - Inria - Institut national de recherche en sciences et technologies du numérique
Conference Papers Year : 2023

Unsupervised space-time network for temporally-consistent segmentation of multiple motions

Abstract

Motion segmentation is one of the main tasks in computer vision and is relevant for many applications. The optical flow (OF) is the input generally used to segment every frame of a video sequence into regions of coherent motion. Temporal consistency is a key feature of motion segmentation, but it is often neglected. In this paper, we propose an original unsupervised spatio-temporal framework for motion segmentation from optical flow that fully investigates the temporal dimension of the problem. More specifically, we have defined a 3D network for multiple motion segmentation that takes as input a sub-volume of successive optical flows and delivers accordingly a sub-volume of coherent segmentation maps. Our network is trained in a fully unsupervised way, and the loss function combines a flow reconstruction term involving spatio-temporal parametric motion models, and a regularization term enforcing temporal consistency on the masks. We have specified an easy temporal linkage of the predicted segments. Besides, we have proposed a flexible and efficient way of coding U-nets. We report experiments on several VOS benchmarks with convincing quantitative results, while not using appearance and not training with any ground-truth data. We also highlight through visual results the distinctive contribution of the short-and long-term temporal consistency brought by our OF segmentation method.
Fichier principal
Vignette du fichier
7630_unsupervised_space_time_network-Camera-ready PDF.pdf (3.41 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04343138 , version 1 (13-12-2023)

Licence

Identifiers

Cite

Etienne Meunier, Patrick Bouthemy. Unsupervised space-time network for temporally-consistent segmentation of multiple motions. CVPR 2023 - IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2023, Vancouver, Canada. pp.1-10, ⟨10.1109/CVPR52729.2023.02120⟩. ⟨hal-04343138⟩
35 View
24 Download

Altmetric

Share

More