Preprints, Working Papers, ... (Preprint) Year : 2026

SENSE: Stereo OpEN Vocabulary SEmantic Segmentation

Abstract

Open-vocabulary semantic segmentation enables models to segment objects or image regions beyond fixed class sets, offering flexibility in dynamic environments. However, existing methods often rely on single-view images and struggle with spatial precision, especially under occlusions and near object boundaries. We propose SENSE, the first work on Stereo OpEN Vocabulary SEmantic Segmentation, which leverages stereo vision and vision-language models to enhance open-vocabulary semantic segmentation. By incorporating stereo image pairs, we introduce geometric cues that improve spatial reasoning and segmentation accuracy. Trained on the PhraseStereo dataset, our approach achieves strong performance in phrase-grounded tasks and demonstrates generalization in zero-shot settings. On PhraseStereo, we show a +2.9% improvement in Average Precision over the baseline method and +0.76% over the best competing method. SENSE also provides a relative improvement of +3.5% mIoU on Cityscapes and +18% on KITTI compared to the baseline work. By jointly reasoning over semantics and geometry, SENSE supports accurate scene understanding from natural language, essential for autonomous robots and Intelligent Transportation Systems.

Fichier principal
Vignette du fichier
main.pdf (2.58 Mo) Télécharger le fichier
Origin Files produced by the author(s)
Licence

Dates and versions

hal-05593870 , version 1 (17-04-2026)

Licence

Identifiers

Cite

Thomas Campagnolo, Ezio Malis, Philippe Martinet, Gaétan Bahl. SENSE: Stereo OpEN Vocabulary SEmantic Segmentation. 2026. ⟨hal-05593870⟩
24 View
9 Download

Altmetric

Share

  • More