On the Correlation of Automatic Audio and Visual Segmentations of Music Videos - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles IEEE Transactions on Circuits and Systems for Video Technology Year : 2007

On the Correlation of Automatic Audio and Visual Segmentations of Music Videos

Abstract

The study of the associations between audio and video content has numerous important applications in the fields of information retrieval and multimedia content authoring. In this work, we focus on music videos which exhibit a broad range of structural and semantic relationships between the music and the video content. To identify such relationships, a two-level automatic struc-turing of the music and the video is achieved separately. Note onsets are detected from the music signal, along with section changes. The latter is achieved by a novel algorithm which makes use of feature selection and statistical novelty detection approaches based on kernel methods. The video stream is independently segmented to detect changes in motion activity, as well as shot boundaries. Based on this two-level segmentation of both streams, four audiovisual correlation measures are computed. The usefulness of these correlation measures is illustrated by a query by video experiment on a 100 music video database, which also exhibits interesting genre dependencies.
No file

Dates and versions

hal-02652635 , version 1 (29-05-2020)

Identifiers

Cite

Olivier Gillet, Slim Essid, Gael Richard. On the Correlation of Automatic Audio and Visual Segmentations of Music Videos. IEEE Transactions on Circuits and Systems for Video Technology, 2007, 17, ⟨10.1109/TCSVT.2007.890831⟩. ⟨hal-02652635⟩
22 View
0 Download

Altmetric

Share

Gmail Facebook X LinkedIn More