Identify, locate and separate: Audio-visual object extraction in large video collections using weak supervision - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2018

Identify, locate and separate: Audio-visual object extraction in large video collections using weak supervision

Sanjeel Parekh
Alexey Ozerov
  • Function : Author
  • PersonId : 882775
Ngoc Q. K. Duong
  • Function : Author
  • PersonId : 946470
Patrick Pérez
  • Function : Author
  • PersonId : 1022281

Abstract

We tackle the problem of audiovisual scene analysis for weakly-labeled data. To this end, we build upon our previous audiovisual representation learning framework to perform object classification in noisy acoustic environments and integrate audio source enhancement capability. This is made possible by a novel use of non-negative matrix factorization for the audio modality. Our approach is founded on the multiple instance learning paradigm. Its effectiveness is established through experiments over a challenging dataset of music instrument performance videos. We also show encouraging visual object localization results.
Fichier principal
Vignette du fichier
weak_nmf_prop.pdf (697.19 Ko) Télécharger le fichier
Origin : Files produced by the author(s)
Loading...

Dates and versions

hal-01914532 , version 1 (07-11-2018)

Identifiers

Cite

Sanjeel Parekh, Alexey Ozerov, Slim Essid, Ngoc Q. K. Duong, Patrick Pérez, et al.. Identify, locate and separate: Audio-visual object extraction in large video collections using weak supervision. 2018. ⟨hal-01914532⟩
96 View
188 Download

Altmetric

Share

Gmail Facebook X LinkedIn More