Summary of A Unified Contrastive Loss For Self-training, by Aurelien Gauffre et al.
A Unified Contrastive Loss for Self-Training
by Aurelien Gauffre, Julien Horvat, Massih-Reza Amini
First submitted to arxiv on: 11 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel framework for enhancing self-training methods in semi-supervised learning, which replaces the traditional cross-entropy loss function (CE) with a unique contrastive loss. By utilizing class prototypes, the framework recovers the probability distributions of the CE setting and establishes a theoretical equivalence between the two. The proposed approach is demonstrated to result in significant performance improvements across three different datasets with limited labeled data, as well as faster convergence speed, better transfer ability, and more stable hyperparameters. The paper also provides code availability at https://github.com/AurelienGauffre/semisupcon/. The contrastive loss function (SupCon) is shown to be particularly effective in exploiting abundant unlabeled data, while unsupervised contrastive learning approaches have been proven to capture high-quality data representations. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper talks about how to make machine learning models work better with less labeled data. It proposes a new way to train models that uses something called “contrastive loss” instead of the usual “cross-entropy loss”. This new approach helps models learn from large amounts of unlabeled data and improves their performance when there’s limited labeled data available. The paper also shows that this method can help models learn faster, transfer well to new tasks, and be more stable with different settings. |
Keywords
» Artificial intelligence » Contrastive loss » Cross entropy » Loss function » Machine learning » Probability » Self training » Semi supervised » Unsupervised
