CertViT: Certified Robustness of Pre-Trained Vision Transformers - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2023

CertViT: Certified Robustness of Pre-Trained Vision Transformers

Résumé

Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks. Code : \url{https://github.com/sagarverma/transformer-lipschitz}
Fichier principal
Vignette du fichier
TransformerLipschitz.pdf (460.87 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03966892 , version 1 (01-02-2023)

Identifiants

  • HAL Id : hal-03966892 , version 1

Citer

Kavya Gupta, Sagar Verma. CertViT: Certified Robustness of Pre-Trained Vision Transformers. 2023. ⟨hal-03966892⟩
58 Consultations
149 Téléchargements

Partager

Gmail Facebook X LinkedIn More