Summary of Self-adaptive Reconstruction with Contrastive Learning For Unsupervised Sentence Embeddings, by Junlong Liu et al.
Self-Adaptive Reconstruction with Contrastive Learning for Unsupervised Sentence Embeddings
by Junlong Liu, Xichen Shang, Huawen Feng, Junhao Zheng, Qianli Ma
First submitted to arxiv on: 23 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The unsupervised sentence embeddings task aims to convert sentences into semantic vector representations. Previous works have relied on pre-trained language models, but these models can be biased towards frequent tokens, leading to poor predictions. To address this issue, a novel framework called Self-Adaptive Reconstruction Contrastive Sentence Embeddings (SARCSE) is proposed. This framework uses an AutoEncoder to reconstruct all tokens in sentences, helping the model to preserve fine-grained semantics during token aggregation. Additionally, a self-adaptive reconstruction loss is introduced to alleviate the bias towards frequency. Experimental results show that SARCSE outperforms the strong baseline SimCSE on 7 STS tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about how to make computers understand sentences better. Right now, computers can’t fully understand sentences because they’re based on words that appear often in language. To fix this, the researchers created a new way of understanding sentences called Self-Adaptive Reconstruction Contrastive Sentence Embeddings (SARCSE). It’s like a special tool that helps computers preserve important details in sentences. This new method works better than the old way, according to tests. |
Keywords
* Artificial intelligence * Autoencoder * Semantics * Token * Unsupervised
