Threshold independent evaluation of sound event detection scores - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Threshold independent evaluation of sound event detection scores

Janek Ebbers
  • Function : Author
  • PersonId : 1125629

Abstract

Performing an adequate evaluation of sound event detection (SED) systems is far from trivial and is still subject to ongoing research. The recently proposed polyphonic sound detection (PSD)-receiver operating characteristic (ROC) and PSD score (PSDS) make an important step into the direction of an evaluation of SED systems which is independent from a certain decision threshold. This allows to obtain a more complete picture of the overall system behavior which is less biased by threshold tuning. Yet, the PSD-ROC is currently only approximated using a finite set of thresholds. The choice of the thresholds used in approximation, however, can have a severe impact on the resulting PSDS. In this paper we propose a method which allows for jointly computing system performance on an evaluation set for all possible thresholds jointly, enabling accurate computation not only of the PSD-ROC and PSDS but also of other collar-based and intersection-based performance curves. It further allows to select the threshold which best fulfills the requirements of a given application. Source code is made publicly available in our SED evaluation package sed scores eval 1 .
Fichier principal
Vignette du fichier
sed_scores_eval_updated.pdf (214.31 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-03562763 , version 1 (09-02-2022)

Identifiers

Cite

Janek Ebbers, Reinhold Haeb-Umbach, Romain Serizel. Threshold independent evaluation of sound event detection scores. ICASSP 2022 - IEEE International Conference on Acoustics, Speech and Signal Processing, May 2022, Singapore, Singapore. ⟨10.1109/ICASSP43922.2022.9747556⟩. ⟨hal-03562763⟩
63 View
196 Download

Altmetric

Share

Gmail Facebook X LinkedIn More