Toward training NLP models to take into account privacy leakages - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2023

Toward training NLP models to take into account privacy leakages


With the rise of machine learning and data-driven models especially in the field of Natural Language Processing (NLP), a strong demand for sharing data between organisations has emerged. However datasets are usually composed of personal data and thus subject to numerous regulations which require anonymization before disseminating the data. In the medical domain for instance, patient records are extremely sensitive and private, but the de-identification of medical documents is a complex task. Recent advances in NLP models have shown encouraging results in this field, but the question of whether deploying such models is safe remains. In this paper, we evaluate three privacy risks on NLP models trained on sensitive data. Specifically, we evaluate counterfactual memorization, which corresponds to rare and sensitive information which has too much influence on the model. We also evaluate membership inference as well as the ability to extract verbatim training data from the model. With this evaluation, we can cure data at risk from the training data and calibrate hyper parameters to provide a supplementary utility and privacy tradeoff to the usual mitigation strategies such as using differential privacy. We exhaustively illustrate the privacy leakage of NLP models through a use-case using medical texts and discuss the impact of both the proposed methodology and mitigation schemes.
Fichier principal
Vignette du fichier
NLP_Privacy_Hopitaux (18).pdf (1.03 Mo) Télécharger le fichier
Origin : Files produced by the author(s)
Licence : Copyright

Dates and versions

hal-04299405 , version 1 (22-11-2023)




  • HAL Id : hal-04299405 , version 1


Gaspard Berthelier, Antoine Boutet, Antoine Richard. Toward training NLP models to take into account privacy leakages. BigData 2023 - IEEE International Conference on Big Data, Dec 2023, Sorrento, Italy. pp.1-9. ⟨hal-04299405⟩
94 View
76 Download


Gmail Facebook X LinkedIn More