Summary of Bcamirs at Semeval-2024 Task 4: Beyond Words: a Multimodal and Multilingual Exploration Of Persuasion in Memes, by Amirhossein Abaskohi et al.
BCAmirs at SemEval-2024 Task 4: Beyond Words: A Multimodal and Multilingual Exploration of Persuasion in Memes
by Amirhossein Abaskohi, Amirhossein Dabiriaghdam, Lele Wang, Giuseppe Carenini
First submitted to arxiv on: 3 Apr 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Computer Vision and Pattern Recognition (cs.CV); Information Theory (cs.IT); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper tackles a challenging problem in natural language processing and computer vision, focusing on identifying persuasive techniques embedded within memes. The authors participated in SemEval-2024 Task 4, a hierarchical multi-label classification task that requires recognizing rhetorical and psychological persuasion methods used in memes. To address this issue, they introduced a caption generation step to assess the impact of semantic information from images on modality gaps. Their best model combines GPT-4 generated captions with meme text to fine-tune RoBERTa as the text encoder and CLIP as the image encoder, outperforming the baseline by a significant margin in all 12 subtasks. This achievement highlights the potential for improving abstract visual semantics encoding. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper explores how to identify persuasive techniques used in memes. Researchers took part in a big challenge called SemEval-2024 Task 4, where they tried to recognize ways that people use to convince others through memes. They came up with an idea to create captions for images and see if it helps understand what’s going on better. Their best approach uses artificial intelligence models like RoBERTa and CLIP, which work together to analyze both text and images in memes. This helps them do a much better job of identifying persuasive techniques than before. |
Keywords
* Artificial intelligence * Classification * Encoder * Gpt * Natural language processing * Semantics
