Summary of Hatesieve: a Contrastive Learning Framework For Detecting and Segmenting Hateful Content in Multimodal Memes, by Xuanyu Su et al.
HateSieve: A Contrastive Learning Framework for Detecting and Segmenting Hateful Content in Multimodal Memes
by Xuanyu Su, Yansong Li, Diana Inkpen, Nathalie Japkowicz
First submitted to arxiv on: 11 Aug 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL); Multimedia (cs.MM); Social and Information Networks (cs.SI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a new framework called HateSieve, designed to detect and segment hateful elements in memes. The rise of Large Multimodal Models (LMMs) has led to concerns about the propagation of biased and harmful content. HateSieve features a Contrastive Meme Generator that creates semantically paired memes, a customized triplet dataset for contrastive learning, and an Image-Text Alignment module for accurate meme segmentation. Experimental results on the Hateful Meme Dataset show that HateSieve outperforms existing LMMs in performance while requiring fewer trainable parameters. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us keep the internet safe by finding and isolating hate speech in memes. Memes are funny pictures or videos with captions, but some of them can be mean-spirited or hurtful. The authors created a new tool called HateSieve to help find these hateful memes and make sure they don’t spread online. They tested their tool on a special dataset of hateful memes and found that it works better than other tools while using fewer computer resources. |
Keywords
» Artificial intelligence » Alignment
