Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles IEEE Journal of Biomedical and Health Informatics Year : 2023

Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning

Abstract

Stuttering is a neuro-developmental speech impairment characterized by uncontrolled utterances (interjections) and core behaviors (blocks, repetitions, and prolongations), and is caused by the failure of speech sensorimotors. Due to its complex nature, stuttering detection (SD) is a difficult task. If detected at an early stage, it could facilitate speech therapists to observe and rectify the speech patterns of persons who stutter (PWS). The stuttered speech of PWS is usually available in limited amounts and is highly imbalanced. To this end, we address the class imbalance problem in the SD domain via a multibranching (MB) scheme and by weighting the contribution of classes in the overall loss function, resulting in a huge improvement in stuttering classes on the SEP-28k dataset over the baseline (StutterNet). To tackle data scarcity, we investigate the effectiveness of data augmentation on top of a multi-branched training scheme. The augmented training outperforms the MB StutterNet (clean) by a relative margin of 4.18% in macro F1-score (F1). In addition, we propose a multi-contextual (MC) StutterNet, which exploits different contexts of the stuttered speech, resulting in an overall improvement of 4.48% in F 1 over the single context based MB StutterNet. Finally, we have shown that applying data augmentation in the cross-corpora scenario can improve the overall SD performance by a relative margin of 13.23% in F1 over the clean training.
Fichier principal
Vignette du fichier
MCDraft.pdf (968.84 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03998392 , version 1 (21-02-2023)

Identifiers

Cite

Shakeel Ahmad Sheikh, Md Sahidullah, Fabrice Hirsch, Slim Ouni. Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learning. IEEE Journal of Biomedical and Health Informatics, 2023, ⟨10.1109/JBHI.2023.3248281⟩. ⟨hal-03998392⟩
92 View
268 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More