Image Semantic Description Based on Deep Learning with Multi-attention Mechanisms - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

Image Semantic Description Based on Deep Learning with Multi-attention Mechanisms

Résumé

In the era of big data, cross-media and multi-modal data are expanding, and data processing methods fail to meet corresponding functional requirements. Aiming at the characteristic of large expression gap of multi-model data, This paper proposes a multimodal data fusion method based on deep learning, which combines the advantages of deep learning in the field of image detection, text sequence prediction, and the multi-attention mechanism. The BLEU algorithm is used to calculate the similarity of four levels of description statements of model output and image. Training and testing were conducted in the Flickr8K data set. Comparing with the traditional single mode state image description method, the experiments show that under the BLEU index, the multi-AM model can achieve better results.
Fichier principal
Vignette du fichier
473854_1_En_36_Chapter.pdf (345.44 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02197765 , version 1 (30-07-2019)

Licence

Paternité

Identifiants

Citer

Jian Yang, Zuqiang Meng. Image Semantic Description Based on Deep Learning with Multi-attention Mechanisms. 10th International Conference on Intelligent Information Processing (IIP), Oct 2018, Nanning, China. pp.356-362, ⟨10.1007/978-3-030-00828-4_36⟩. ⟨hal-02197765⟩
31 Consultations
71 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More