PDF-Distil: including Prediction Disagreements in Feature-based Distillation for object detection - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

PDF-Distil: including Prediction Disagreements in Feature-based Distillation for object detection

Heng Zhang
  • Function : Author
  • PersonId : 1107306
Elisa Fromont
Bruno Avignon
  • Function : Author
  • PersonId : 1120804

Abstract

Knowledge distillation aims at compressing deep models by transferring the learned knowledge from precise but cumbersome teacher models to compact student models. Due to the extreme imbalance between the foreground and the background of images, when traditional knowledge distillation methods are directly applied to the object detection task, there is a large performance gap between the teacher model and the student model. We tackle this imbalance problem from a sampling perspective, and we propose to include the teacher-student prediction disagreements into a feature-based detection distillation framework. This is done with PDF-Distil by dynamically generating a weighting mask applied to the knowledge distillation loss, based on the disagreements between the predictions of both models. Extensive experiments on PASCAL VOC and MS COCO datasets demonstrate that, compared to state-of-the-art methods, PDF-Distil is able to better reduce the performance gap between the teacher and student models.
Fichier principal
Vignette du fichier
BMVC_Distil.pdf (2.9 Mo) Télécharger le fichier
Origin Publisher files allowed on an open archive

Dates and versions

hal-03487128 , version 1 (17-12-2021)

Identifiers

  • HAL Id : hal-03487128 , version 1

Cite

Heng Zhang, Elisa Fromont, Sébastien Lefèvre, Bruno Avignon. PDF-Distil: including Prediction Disagreements in Feature-based Distillation for object detection. BMVC 2021 - 32nd British Machine Vision Conference, Nov 2021, Virtual, United Kingdom. pp.1-13. ⟨hal-03487128⟩
110 View
90 Download

Share

Gmail Mastodon Facebook X LinkedIn More