A comparative study of different state-of-the-art NLP models for efficient automatic hate speech detection - Inria - Institut national de recherche en sciences et technologies du numérique Access content directly
Conference Papers Year : 2021

A comparative study of different state-of-the-art NLP models for efficient automatic hate speech detection

Abstract

Hate speech (HS) is legally punished in many countries. Manual moderation of hate messages on social networks is no longer possible due to the huge number of messages posted every day. Automatic methods are needed to remove harmful messages. In this article, we are interested on HS detection (HSD) in social media (Twitter). Hateful content is more than just keyword detection. Hate may be implied, tweets can be grammatically incorrect and the abbreviations and slangs may be numerous. In this condition, the HSD is a complex task.Recently, natural language processing (NLP) methods have been proposed for the detection of HS. In particular, systems based on deep neural networks (DNN) have been shown to have notable performance for HSD.In this paper, we study different state-of-the-art NLP models for the HSD task. Indeed, powerful new models based on transformers have recently emerged in the literature, such as BERT, HateBERT, BERTweet, SemBERT and USE (Universal Sentence Encoder). These models are trained on different generic corpora collected from various sources. These BERT-based models can be fine-tuned for a specific task. Some models have particularities. The BERT, HateBERT and BERTweet models can be used to extract features at words or sentence level. The SemBERT model models only word-level features and further incorporate explicit contextual semantics. The USE model generates sentence-level features. The HateBERT model is trained on a Reddit corpus with a high potential of hateful content. The BERTweet model is trained on more than 840M tweets. The goal of this article is to study the generalizability of these models on HSD in tweets and to investigate the impact of sentence-level and word-level features.We investigate different DNN structures for HSD using the transformer-based models. The impact of word and sentence-based methods was assessed. Our experiments were performed on two HS corpora extracted from Twitter: Founta and Davidson (Founta et al, 2018; Davidson et al, 2017). The best performances were obtained with the BERTweet model for the two corpora (77.3% and 78.5% macro-F1 on the Founta and Davidson test sets respectively). This work was performed in the context of the French-German ANR-DFG project M-PHASIS.
No file

Dates and versions

hal-03347244 , version 1 (17-09-2021)

Identifiers

  • HAL Id : hal-03347244 , version 1

Cite

Nicolas Zampieri, Irina Illina, Dominique Fohr. A comparative study of different state-of-the-art NLP models for efficient automatic hate speech detection. Comments, hate speech, disinformation and public communication regulation 2021, Sep 2021, Zagreb, Croatia. ⟨hal-03347244⟩
323 View
1 Download

Share

Gmail Mastodon Facebook X LinkedIn More