CertViT: Certified Robustness of Pre-Trained Vision Transformers - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Preprints, Working Papers, ... Year : 2023

CertViT: Certified Robustness of Pre-Trained Vision Transformers

Abstract

Lipschitz bounded neural networks are certifiably robust and have a good trade-off between clean and certified accuracy. Existing Lipschitz bounding methods train from scratch and are limited to moderately sized networks (< 6M parameters). They require a fair amount of hyper-parameter tuning and are computationally prohibitive for large networks like Vision Transformers (5M to 660M parameters). Obtaining certified robustness of transformers is not feasible due to the non-scalability and inflexibility of the current methods. This work presents CertViT, a two-step proximal-projection method to achieve certified robustness from pre-trained weights. The proximal step tries to lower the Lipschitz bound and the projection step tries to maintain the clean accuracy of pre-trained weights. We show that CertViT networks have better certified accuracy than state-of-the-art Lipschitz trained networks. We apply CertViT on several variants of pre-trained vision transformers and show adversarial robustness using standard attacks. Code : \url{https://github.com/sagarverma/transformer-lipschitz}
Fichier principal
Vignette du fichier
TransformerLipschitz.pdf (460.87 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03966892 , version 1 (01-02-2023)

Identifiers

  • HAL Id : hal-03966892 , version 1

Cite

Kavya Gupta, Sagar Verma. CertViT: Certified Robustness of Pre-Trained Vision Transformers. 2023. ⟨hal-03966892⟩
58 View
150 Download

Share

Gmail Mastodon Facebook X LinkedIn More