Embedding Space Interpolation Beyond Mini-Batch, Beyond Pairs and Beyond Examples - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Embedding Space Interpolation Beyond Mini-Batch, Beyond Pairs and Beyond Examples

Résumé

Mixup refers to interpolation-based data augmentation, originally motivated as a way to go beyond empirical risk minimization (ERM). Its extensions mostly focus on the definition of interpolation and the space (input or embedding) where it takes place, while the augmentation process itself is less studied. In most methods, the number of generated examples is limited to the mini-batch size and the number of examples being interpolated is limited to two (pairs), in the input space. We make progress in this direction by introducing MultiMix, which generates an arbitrarily large number of interpolated examples beyond the mini-batch size, and interpolates the entire mini-batch in the embedding space. Effectively, we sample on the entire convex hull of the mini-batch rather than along linear segments between pairs of examples. On sequence data we further extend to Dense MultiMix. We densely interpolate features and target labels at each spatial location and also apply the loss densely. To mitigate the lack of dense labels, we inherit labels from examples and weight interpolation factors by attention as a measure of confidence. Overall, we increase the number of loss terms per mini-batch by orders of magnitude at little additional cost. This is only possible because of interpolating in the embedding space. We empirically show that our solutions yield significant improvement over state-of-the-art mixup methods on four different benchmarks, despite interpolation being only linear. By analyzing the embedding space, we show that the classes are more tightly clustered and uniformly spread over the embedding space, thereby explaining the improved behavior.
Fichier principal
Vignette du fichier
MultiMix_NeurIPS23.pdf (1.59 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04214672 , version 1 (22-09-2023)

Licence

Paternité

Identifiants

  • HAL Id : hal-04214672 , version 1

Citer

Shashanka Venkataramanan, Ewa Kijak, Laurent Amsaleg, Yannis Avrithis. Embedding Space Interpolation Beyond Mini-Batch, Beyond Pairs and Beyond Examples. NeurIPS 2023 - 37th Conference on Neural Information Processing Systems, Dec 2023, New Orleans (Louisiana), United States. pp.1-17. ⟨hal-04214672⟩
86 Consultations
111 Téléchargements

Partager

Gmail Facebook X LinkedIn More