Post-Processing Independent Evaluation of Sound Event Detection Systems - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Post-Processing Independent Evaluation of Sound Event Detection Systems

Janek Ebbers
  • Function : Author
  • PersonId : 1125629

Abstract

Due to the high variation in the application requirements of sound event detection (SED) systems, it is not sufficient to evaluate systems only in a single operating point. Therefore, the community recently adopted the polyphonic sound detection score (PSDS) as an evaluation metric, which is the normalized area under the PSD-ROC. It summarizes the system performance over a range of operating points. Hence, it provides a more complete picture of the overall system behavior and is less biased by hyper parameter tuning. So far PSDS has only been computed over operating points resulting from varying the decision threshold that is used to translate the system output scores into a binary detection output. However, besides the decision threshold there is also the post-processing that can be changed to enter another operating mode. In this paper we propose the post-processing independent PSDS (piPSDS) which computes PSDS over operating points with varying post-processings and varying decision thresholds. It summarizes even more operating modes of an SED system and allows for system comparison without the need of implementing a post-processing and without a bias due to different post-processings. While piPSDS can in principle also combine different types of post-processing, we here, as a first step, present median filter independent PSDS (miPSDS) results for this year's DCASE Challenge Task4a systems. Source code is publicly available in our sed scores eval package 1 .
Fichier principal
Vignette du fichier
Post_Processing_Independent_Evaluation_of_Sound_Event_Detection_Systems.pdf (218.22 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04385022 , version 1 (10-01-2024)

Licence

Identifiers

Cite

Janek Ebbers, Reinhold Haeb-Umbach, Romain Serizel. Post-Processing Independent Evaluation of Sound Event Detection Systems. DCASE 2023 - 8th Workshop on Detection and Classification of Acoustic Scenes and Events, Sep 2023, Tampere, Finland. ⟨10.48550/arXiv.2306.15440⟩. ⟨hal-04385022⟩
21 View
13 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More