Low-cost Multispectral Scene Analysis with Modality Distillation - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2022

Low-cost Multispectral Scene Analysis with Modality Distillation


Despite its robust performance under various illumination conditions, multispectral scene analysis has not been widely deployed due to two strong practical limitations: 1) thermal cameras, especially high-resolution ones are much more expensive than conventional visible cameras; 2) the most commonly adopted multispectral architectures, twostream neural networks, nearly double the inference time of a regular mono-spectral model which makes them impractical in embedded environments. In this work, we aim to tackle these two limitations by proposing a novel knowledge distillation framework named Modality Distillation (MD). The proposed framework distils the knowledge from a high thermal resolution two-stream network with featurelevel fusion to a low thermal resolution one-stream network with image-level fusion. We show on different multispectral scene analysis benchmarks that our method can effectively allow the use of low-resolution thermal sensors with more compact one-stream networks.
Fichier principal
Vignette du fichier
WACV2022_MD.pdf (7.13 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03491950 , version 1 (18-12-2021)


  • HAL Id : hal-03491950 , version 1


Heng Zhang, Elisa Fromont, Sébastien Lefèvre, Bruno Avignon. Low-cost Multispectral Scene Analysis with Modality Distillation. WACV 2022 - IEEE Winter Conference on Applications of Computer Vision, Jan 2022, Waikoloa /Virtual, United States. pp.1-10. ⟨hal-03491950⟩
36 View
46 Download


Gmail Mastodon Facebook X LinkedIn More