Learning Semantic and Visual Similarity for Endomicroscopy Video Retrieval - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Reports (Research Report) Year : 2011

Learning Semantic and Visual Similarity for Endomicroscopy Video Retrieval


Traditional Content-Based Image Retrieval (CBIR) systems only deliver visual outputs that are not directly interpretable by the physicians. Our objective is to provide a system for endomicroscopy video retrieval which delivers both visual and semantic outputs that are consistent with each other. In a previous study, we developed an adapted bag-of-visual-words method for endomicroscopy retrieval that computes a visual signature for each video. In this study, we first leverage semantic ground-truth data to transform these visual signatures into semantic signatures that reflect how much the presence of each semantic concept is expressed by the visual words describing the videos. Using cross-validation, we demonstrate that our visual-word-based semantic signatures enable a recall performance which is significantly higher than those of several state-of-the-art methods in CBIR. In a second step, we propose to improve retrieval relevance by learning, from a perceived similarity ground truth, an adjusted similarity distance. Our distance learning method allows to improve, with statistical significance, the correlation with the perceived similarity. Our resulting retrieval system is efficient in providing both visual and semantic information that are correlated with each other and clinically interpretable by the endoscopists.
Fichier principal
Vignette du fichier
RR-7722.pdf (909.06 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

inria-00618057 , version 1 (31-08-2011)


  • HAL Id : inria-00618057 , version 1


Barbara André, Tom Vercauteren, Anna M. Buchner, Michael B. Wallace, Nicholas Ayache. Learning Semantic and Visual Similarity for Endomicroscopy Video Retrieval. [Research Report] RR-7722, INRIA. 2011. ⟨inria-00618057⟩
242 View
432 Download


Gmail Facebook Twitter LinkedIn More