On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks

Abstract

Graph Neural Networks (GNNs) have succeeded in various computer science applications, yet deep GNNs underperform their shallow counterparts despite deep learning's success in other domains. Over-smoothing and over-squashing are key challenges when stacking graph convolutional layers, hindering deep representation learning and information propagation from distant nodes. Our work reveals that over-smoothing and over-squashing are intrinsically related to the spectral gap of the graph Laplacian, resulting in an inevitable trade-off between these two issues, as they cannot be alleviated simultaneously. To achieve a suitable compromise, we propose adding and removing edges as a viable approach. We introduce the Stochastic Jost and Liu Curvature Rewiring (SJLR) algorithm, which is computationally efficient and preserves fundamental properties compared to previous curvature-based methods. Unlike existing approaches, SJLR performs edge addition and removal during GNN training while maintaining the graph unchanged during testing. Comprehensive comparisons demonstrate SJLR's competitive performance in addressing over-smoothing and over-squashing. CCS CONCEPTS • Computing methodologies → Machine learning algorithms; • Computer systems organization → Neural networks.
Fichier principal
Vignette du fichier
CIKM_2023_Camera_Ready.pdf (878.09 Ko) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

hal-04263891 , version 1 (29-10-2023)

Identifiers

Cite

Jhony H Giraldo, Konstantinos Skianis, Thierry Bouwmans, Fragkiskos D. Malliaros. On the Trade-off between Over-smoothing and Over-squashing in Deep Graph Neural Networks. CIKM 2023 - The 32nd ACM International Conference on Information and Knowledge Management, Oct 2023, Birmingham United Kingdom, United Kingdom. pp.566-576, ⟨10.1145/3583780.3614997⟩. ⟨hal-04263891⟩
73 View
121 Download

Altmetric

Share

Gmail Facebook X LinkedIn More