MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Journal Articles Transactions on Machine Learning Research Journal Year : 2024

MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks

Felix Hensel
  • Function : Author
  • PersonId : 1125447
Charles Arnal
Mathieu Carrière
Yuichi Ike
  • Function : Author
Frédéric Chazal

Abstract

Despite their successful application to a variety of tasks, neural networks remain limited, like other machine learning methods, by their sensitivity to shifts in the data: their performance can be severely impacted by differences in distribution between the data on which they were trained and that on which they are deployed. In this article, we propose a new family of representations, called MAGDiff, that we extract from any given neural network classifier and that allows for efficient covariate data shift detection without the need to train a new model dedicated to this task. These representations are computed by comparing the activation graphs of the neural network for samples belonging to the training distribution and to the target distribution, and yield powerful data- and task-adapted statistics for the two-sample tests commonly used for data set shift detection. We demonstrate this empirically by measuring the statistical powers of two-sample Kolmogorov-Smirnov (KS) tests on several different data sets and shift types, and showing that our novel representations induce significant improvements over a state-of-the-art baseline relying on the network output.

Dates and versions

hal-04103272 , version 1 (23-05-2023)

Licence

Identifiers

Cite

Felix Hensel, Charles Arnal, Mathieu Carrière, Théo Lacombe, Hiroaki Kurihara, et al.. MAGDiff: Covariate Data Set Shift Detection via Activation Graphs of Deep Neural Networks. Transactions on Machine Learning Research Journal, 2024. ⟨hal-04103272⟩
48 View
0 Download

Altmetric

Share

Gmail Mastodon Facebook X LinkedIn More