Joint processing of linguistic properties in brains and language models - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Joint processing of linguistic properties in brains and language models

Résumé

Language models have been shown to be very effective in predicting brain recordings of subjects experiencing complex language stimuli. For a deeper understanding of this alignment, it is important to understand the correspondence between the detailed processing of linguistic information by the human brain versus language models. We investigate this correspondence via a direct approach, in which we eliminate information related to specific linguistic properties in the language model representations and observe how this intervention affects the alignment with fMRI brain recordings obtained while participants listened to a story. We investigate a range of linguistic properties (surface, syntactic, and semantic) and find that the elimination of each one results in a significant decrease in brain alignment. Specifically, we find that syntactic properties (i.e. Top Constituents and Tree Depth) have the largest effect on the trend of brain alignment across model layers. These findings provide clear evidence for the role of specific linguistic information in the alignment between brain and language models, and open new avenues for mapping the joint information processing in both systems. We make the code publicly available.
Fichier principal
Vignette du fichier
2212.08094.pdf (8.67 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04400635 , version 1 (17-01-2024)

Licence

Paternité

Identifiants

  • HAL Id : hal-04400635 , version 1

Citer

Subba Reddy Oota, Manish Gupta, Mariya Toneva. Joint processing of linguistic properties in brains and language models. NeurIPS 2023 - 37th Conference on Neural Information Processing Systems, Neural Information Processing Systems, Dec 2023, New Orleans, United States. ⟨hal-04400635⟩

Collections

CNRS INRIA INRIA2
10 Consultations
5 Téléchargements

Partager

Gmail Facebook X LinkedIn More