Summary of Robustsentembed: Robust Sentence Embeddings Using Adversarial Self-supervised Contrastive Learning, by Javad Rafiei Asl et al.
RobustSentEmbed: Robust Sentence Embeddings Using Adversarial Self-Supervised Contrastive Learning
by Javad Rafiei Asl, Prajwal Panzade, Eduardo Blanco, Daniel Takabi, Zhipeng Cai
First submitted to arxiv on: 17 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces RobustSentEmbed, a self-supervised sentence embedding framework that improves both generalization and robustness in diverse text representation tasks against various adversarial attacks. By generating high-risk adversarial perturbations and utilizing them in a novel objective function, RobustSentEmbed learns high-quality and robust sentence embeddings. The proposed framework outperforms state-of-the-art representations, reducing the success rate of BERTAttack by almost half (from 75.51% to 38.81%). Additionally, it yields improvements in semantic textual similarity tasks and transfer tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a special kind of computer model that helps us understand text better. It’s called RobustSentEmbed, and it makes the model more robust to tricky attacks that try to make it make mistakes. The model uses some clever tricks to generate fake versions of sentences that are meant to confuse it, but instead, they help the model learn to be more accurate and resistant to these types of attacks. This new model is better than older models at understanding text and performing tasks like comparing sentences for meaning. |
Keywords
* Artificial intelligence * Embedding * Generalization * Objective function * Self supervised
