GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph

Noa Garcia
  • Function : Author
  • PersonId : 1098949
Chenhui Chu
  • Function : Author
  • PersonId : 1098950
Yuta Nakashima
  • Function : Author
  • PersonId : 1098951
Hajime Nagahara
  • Function : Author
  • PersonId : 1098952

Abstract

The rise of digitization of cultural documents offers large-scale contents, opening the road for development of AI systems in order to preserve, search, and deliver cultural heritage. To organize such cultural content also means to classify them, a task that is very familiar to modern computer science. Contextual information is often the key to structure such real world data, and we propose to use it in form of a knowledge graph. Such a knowledge graph, combined with content analysis, enhances the notion of proximity between artworks so it improves the performances in classification tasks. In this paper, we propose a novel use of a knowledge graph, that is constructed on annotated data and pseudo-labeled data. With label propagation, we boost artwork classification by training a model using a graph convolutional network, relying on the relationships between entities of the knowledge graph. Following a transductive learning framework, our experiments show that relying on a knowledge graph modeling the relations between labeled data and unlabeled data allows to achieve state-of-the-art results on multiple classification tasks on a dataset of paintings, and on a dataset of Buddha statues. Additionally, we show state-of-the-art results for the difficult case of dealing with unbalanced data, with the limitation of disregarding classes with extremely low degrees in the knowledge graph.
Fichier principal
Vignette du fichier
GCNBoost-hal.pdf (5.31 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03228787 , version 1 (18-05-2021)

Identifiers

  • HAL Id : hal-03228787 , version 1

Cite

Cheikh Brahim El Vaigh, Noa Garcia, Benjamin Renoust, Chenhui Chu, Yuta Nakashima, et al.. GCNBoost: Artwork Classification by Label Propagation through a Knowledge Graph. ICMR 2021 - ACM International Conference on Multimedia Retrieval, Aug 2021, Taipei, Taiwan. ⟨hal-03228787⟩
110 View
163 Download

Share

Gmail Mastodon Facebook X LinkedIn More