Towards Weakly-Supervised Action Localization - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2016

Towards Weakly-Supervised Action Localization

Résumé

This paper presents a novel approach for weakly-supervised action localization, i.e., that does not require per-frame spatial annotations for training. We first introduce an effective method for extracting human tubes by combining a state-of-the-art human detector with a tracking-by-detection approach. Our tube extraction leverages the large amount of annotated humans available today and outperforms the state of the art by an order of magnitude: with less than 5 tubes per video, we obtain a recall of 95% on the UCF-Sports and J-HMDB datasets. Given these human tubes, we perform weakly-supervised selection based on multi-fold Multiple Instance Learning (MIL) with improved dense trajectories and achieve excellent results. We obtain a mAP of 84% on UCF-Sports, 54% on J-HMDB and 45% on UCF-101, which outperforms the state of the art for weakly-supervised action localization and is close to the performance of the best fully-supervised approaches. The second contribution of this paper is a new realistic dataset for action localization, named DALY (Daily Action Localization in YouTube). It contains high quality temporal and spatial annotations for 10 actions in 31 hours of videos (3.3M frames), which is an order of magnitude larger than standard action localization datasets. On the DALY dataset, our tubes have a spatial recall of 82%, but the detection task is extremely challenging, we obtain 10.8% mAP.
Fichier principal
Vignette du fichier
weak_action.pdf (7.62 Mo) Télécharger le fichier
Vignette du fichier
weak_action.jpg (116.2 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Format : Figure, Image
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01317558 , version 1 (18-05-2016)
hal-01317558 , version 2 (24-05-2017)

Identifiants

Citer

Philippe Weinzaepfel, Xavier Martin, Cordelia Schmid. Towards Weakly-Supervised Action Localization. 2016. ⟨hal-01317558v1⟩
1137 Consultations
1502 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More