Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer

Abstract

Recent research has shown that independently trained encoders and decoders, combined through a shared fixed-size representation, can achieve competitive performance in speech-to-text translation. In this work, we show that this type of approach can be further improved with multilingual training. We observe significant improvements in zero-shot cross-modal speech translation, even outperforming a supervised approach based on XLSR for several languages.
Fichier principal
Vignette du fichier
Modular_speech_to_text_translation_for_zero_shot_cross_modal_transfer.pdf (206.82 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04264023 , version 1 (29-10-2023)

Identifiers

Cite

Paul-Ambroise Duquenne, Holger Schwenk, Benoît Sagot. Modular Speech-to-Text Translation for Zero-Shot Cross-Modal Transfer. INTERSPEECH 2023, Aug 2023, Dublin, Ireland. ⟨10.21437/Interspeech.2023-2484⟩. ⟨hal-04264023⟩
9 View
40 Download

Altmetric

Share

Gmail Facebook X LinkedIn More