Summary of Attentive Fusion: a Transformer-based Approach to Multimodal Hate Speech Detection, by Atanu Mandal et al.
Attentive Fusion: A Transformer-based Approach to Multimodal Hate Speech Detection
by Atanu Mandal, Gargi Roy, Amit Barman, Indranil Dutta, Sudip Kumar Naskar
First submitted to arxiv on: 19 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel approach to identifying hate speech in social media content, addressing the limitations of solely relying on audio or text-based analysis. The authors develop a methodology that leverages both audio and textual representations using the Transformer framework and their proprietary “Attentive Fusion” layer. By combining these two modalities, the model achieves a state-of-the-art macro F1 score of 0.927 on the test set. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research is important because it helps us understand how to identify hateful messages on social media. Right now, people are using social media more and more, and we need ways to stop hate speech from spreading. Traditionally, researchers have focused on text-based content, but that’s not enough. Some people use sarcasm in their writing or speaking, which makes it harder to detect hate speech. The authors of this paper came up with a new way to identify hate speech by using both audio and text together. This is a big step forward in stopping the spread of hateful messages. |
Keywords
* Artificial intelligence * F1 score * Transformer
